Skip to content

MINIFICPP-2863 - Introduce Reporting Tasks - #2231

Open
adamdebreceni wants to merge 5 commits into
apache:MINIFICPP-2845from
adamdebreceni:MINIFICPP-2863
Open

MINIFICPP-2863 - Introduce Reporting Tasks#2231
adamdebreceni wants to merge 5 commits into
apache:MINIFICPP-2845from
adamdebreceni:MINIFICPP-2863

Conversation

@adamdebreceni

Copy link
Copy Markdown
Contributor

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you to ensure the following steps have been taken:

For all changes:

  • Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

  • Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

  • Has your PR been rebased against the latest commit within the target branch (typically main)?

  • Is your initial contribution a single, squashed commit?

For code changes:

  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE file?
  • If applicable, have you updated the NOTICE file?

For documentation related changes:

  • Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.

@adamdebreceni
adamdebreceni requested a lite review from Copilot August 5, 2026 14:19
@adamdebreceni
adamdebreceni deleted the branch apache:MINIFICPP-2845 August 5, 2026 14:28
@adamdebreceni adamdebreceni reopened this Aug 5, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Introduces first-class Reporting Tasks to MiNiFi C++ by adding API abstractions, class-loading support, and flow parsing so reporting tasks can be defined and executed from flow configuration (including a Site-to-Site provenance reporting task).

Changes:

  • Added reporting task API types (context/descriptor/factory/metadata) and wired them into the class loader/resource registration system.
  • Replaced legacy “Provenance Reporting” flow section with generalized “Reporting Tasks” parsing (YAML + NiFi Flow JSON).
  • Updated Site-to-Site provenance reporting implementation and tests/behave framework to use the new reporting task model.

Reviewed changes

Copilot reviewed 44 out of 44 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
minifi-api/include/minifi-cpp/provenance/Provenance.h API change to provenance send() signature used by reporting/task flow changes
minifi-api/include/minifi-cpp/core/reporting/ReportingTaskMetadata.h New reporting task metadata struct for factory construction
minifi-api/include/minifi-cpp/core/reporting/ReportingTaskFactory.h New factory interface for reporting tasks
minifi-api/include/minifi-cpp/core/reporting/ReportingTaskDescriptor.h New descriptor interface to declare supported properties
minifi-api/include/minifi-cpp/core/reporting/ReportingTaskContext.h New context interface to access properties/services/repos/state for reporting tasks
minifi-api/include/minifi-cpp/core/reporting/ReportingTaskApi.h New reporting task lifecycle API
minifi-api/include/minifi-cpp/core/ProcessContext.h Makes ProcessContext implement ReportingTaskContext (shared context surface)
minifi-api/include/minifi-cpp/core/ClassLoader.h Adds class loader registration overload for reporting task factories
minifi-api/include/minifi-cpp/agent/agent_docs.h Adds ReportingTask to resource type taxonomy
minifi-api/include/minifi-api.h Extends C API surface with reporting task metadata/callbacks/context property getter
libminifi/test/unit/LogMetricsPublisherTests.cpp Adjusts fixture resource initialization/teardown to match repo directory requirements
libminifi/test/unit/JsonFlowSerializerTests.cpp Adds JSON parsing test coverage for reportingTasks
libminifi/test/libtest/unit/ReportingTaskUtils.h Adds helpers to instantiate reporting tasks for unit tests
libminifi/src/sitetosite/SiteToSiteClient.cpp Replaces ProcessContext& yielding with YieldAction adapter
libminifi/src/sitetosite/RawSiteToSiteClient.cpp Updates payload transmission to use YieldAction
libminifi/src/sitetosite/HttpSiteToSiteClient.cpp Updates interface implementation to new YieldAction signature
libminifi/src/provenance/Provenance.cpp Changes provenance commit error handling + updates send() behavior
libminifi/src/core/reporting/SiteToSiteProvenanceReportingTask.cpp Migrates provenance reporting task to new reporting task APIs
libminifi/src/core/reporting/ReportingTask.cpp Implements processor wrapper that runs ReportingTaskApi instances
libminifi/src/core/flow/StructuredConfiguration.cpp Parses and instantiates reporting tasks from flow config
libminifi/src/core/flow/FlowSchema.cpp Adds schema keys for reporting tasks and removes legacy provenance reporting keys
libminifi/src/core/FlowConfiguration.cpp Removes legacy provenance report task factory method
libminifi/src/core/ClassLoader.cpp Adds reporting task factory wrapper and registration plumbing
libminifi/src/RemoteProcessGroupPort.cpp Adds useProtocol() helper + decouples protocol pool sizing from ProcessContext
libminifi/include/sitetosite/SiteToSiteClient.h Introduces YieldAction type-erased yield adapter + updates transmit API
libminifi/include/sitetosite/RawSiteToSiteClient.h Updates transmit signature to YieldAction
libminifi/include/sitetosite/HttpSiteToSiteClient.h Updates transmit signature to YieldAction
libminifi/include/provenance/Provenance.h Updates ProvenanceReporterImpl::send() override signature
libminifi/include/core/reporting/SiteToSiteProvenanceReportingTask.h Converts provenance reporting to ReportingTaskBase and embeds an RPG port
libminifi/include/core/reporting/ReportingTask.h Adds processor-facing wrapper for reporting tasks
libminifi/include/core/flow/StructuredConfiguration.h Exposes parseReportingTasks() and removes legacy provenance reporting parser
libminifi/include/core/flow/FlowSchema.h Adds schema fields for reporting tasks
libminifi/include/core/FlowConfiguration.h Removes legacy provenance report task creation API
libminifi/include/RemoteProcessGroupPort.h Adds reporting-context onSchedule overload + useProtocol()
extensions/standard-processors/tests/unit/ProcessorTests.cpp Updates tests to build/report on new reporting-task wrapper
extensions/standard-processors/tests/features/reporting-tasks.feature Adds behave feature for executing reporting tasks end-to-end
extensions/rocksdb-repos/tests/ProvenanceTests.cpp Updates tests to assert append success (new return-value expectations)
core-framework/include/core/reporting/ReportingTaskFactoryImpl.h Provides templated factory implementation for reporting tasks
core-framework/include/core/reporting/ReportingTaskBase.h Provides base class with descriptor/property registration helpers
core-framework/include/core/Resource.h Adds reporting task resource registration to static resource mechanism
behave_framework/src/minifi_behave/steps/flow_building_steps.py Adds steps to assemble flows containing reporting tasks
behave_framework/src/minifi_behave/minifi/reporting_task.py Adds reporting-task model + YAML serialization
behave_framework/src/minifi_behave/minifi/minifi_flow_definition.py Emits “Reporting Tasks” section in generated YAML
behave_framework/src/minifi_behave/minifi/flow_definition.py Tracks reporting tasks in the in-memory flow definition
Suppressed comments (6)

minifi-api/include/minifi-cpp/core/reporting/ReportingTaskContext.h:1

  • ReportingTaskContext.h is not self-contained: it uses Property, PropertyReference, core::controller::ControllerServiceHandle, and StateManager without including the required headers or providing forward declarations (and the inline getProperty(...) overloads require full definitions). This can break consumers that include this header directly. Add the appropriate includes (e.g., property headers and ControllerServiceHandle/StateManager declarations) or restructure to avoid inline member definitions that depend on incomplete types.
    minifi-api/include/minifi-cpp/core/reporting/ReportingTaskContext.h:1
  • ReportingTaskContext.h is not self-contained: it uses Property, PropertyReference, core::controller::ControllerServiceHandle, and StateManager without including the required headers or providing forward declarations (and the inline getProperty(...) overloads require full definitions). This can break consumers that include this header directly. Add the appropriate includes (e.g., property headers and ControllerServiceHandle/StateManager declarations) or restructure to avoid inline member definitions that depend on incomplete types.
    minifi-api/include/minifi-cpp/core/reporting/ReportingTaskContext.h:1
  • ReportingTaskContext.h is not self-contained: it uses Property, PropertyReference, core::controller::ControllerServiceHandle, and StateManager without including the required headers or providing forward declarations (and the inline getProperty(...) overloads require full definitions). This can break consumers that include this header directly. Add the appropriate includes (e.g., property headers and ControllerServiceHandle/StateManager declarations) or restructure to avoid inline member definitions that depend on incomplete types.
    minifi-api/include/minifi-cpp/provenance/Provenance.h:1
  • This changes the public ProvenanceReporter API by removing the force parameter from send(...), which is a breaking change for downstream implementations/extensions. If backward compatibility is required, consider keeping the old signature (possibly deprecated) and forwarding to the new one (or providing a defaulted parameter), or clearly version/bump the API boundary to avoid silently breaking external implementers.
    minifi-api/include/minifi-api.h:1
  • Two documentation issues in the new C API additions: (1) minifi_reporting_task_metadata.logger comment refers to 'controller service' lifetime instead of 'reporting task'; (2) the full_name example string is missing its closing quote, making the comment misleading for API consumers. Update these comments to match reporting-task terminology and provide a syntactically correct example.
    minifi-api/include/minifi-api.h:1
  • Two documentation issues in the new C API additions: (1) minifi_reporting_task_metadata.logger comment refers to 'controller service' lifetime instead of 'reporting task'; (2) the full_name example string is missing its closing quote, making the comment misleading for API consumers. Update these comments to match reporting-task terminology and provide a syntactically correct example.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +458 to +460
if (auto append_status = repo_->appendEvents(events_); !append_status) {
throw minifi::Exception(REPOSITORY_EXCEPTION, append_status.error());
}
Comment on lines +37 to +41
explicit SiteToSiteProvenanceReportingTask(ReportingTaskMetadata metadata)
: ReportingTaskBase{metadata},
remote_port_{metadata.name, metadata.uuid, std::make_unique<RemoteProcessGroupPort>(metadata.name, "", Configure::create(),
metadata.uuid, sitetosite::TransferDirection::SEND, metadata.logger)}
{
Comment on lines +1088 to +1101
auto scheduling_strategy = getOptionalField(reporting_task_node, schema_.scheduling_strategy, DEFAULT_SCHEDULING_STRATEGY);
if (scheduling_strategy == "TIMER_DRIVEN") {
reporting_task->setSchedulingStrategy(core::TIMER_DRIVEN);
} else {
reporting_task->setSchedulingStrategy(core::CRON_DRIVEN);
}
auto scheduling_period_str = getOptionalField(reporting_task_node, schema_.scheduling_period, DEFAULT_SCHEDULING_PERIOD_STR);
if (scheduling_strategy == "TIMER_DRIVEN") {
if (auto scheduling_period = utils::timeutils::StringToDuration<std::chrono::nanoseconds>(scheduling_period_str)) {
reporting_task->setSchedulingPeriod(*scheduling_period);
}
} else {
reporting_task->setCronPeriod(scheduling_period_str);
}
Comment on lines +1127 to +1129
if (Node properties_node = reporting_task_node[schema_.processor_properties]) {
parsePropertiesNode(properties_node, *reporting_task, name, nullptr);
}
Node parameterContextsNode = root_node[schema_.parameter_contexts];
Node parameterProvidersNode = root_node[schema_.parameter_providers];
Node provenanceReportNode = root_node[schema_.provenance_reporting];
Node reportingTasksNode = root_node[schema_.reporting_tasks];
std::unique_ptr<core::ProcessGroup> root = parseRootProcessGroup(root_node);
parseProvenanceReporting(provenanceReportNode, root.get());

parseReportingTasks(reportingTasksNode, root.get());
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants