Skip to content

OCPBUGS-74872: Sort plugin list to make them deterministic#1107

Open
jhadvig wants to merge 1 commit into
openshift:mainfrom
jhadvig:OCPBUGS-74872
Open

OCPBUGS-74872: Sort plugin list to make them deterministic#1107
jhadvig wants to merge 1 commit into
openshift:mainfrom
jhadvig:OCPBUGS-74872

Conversation

@jhadvig
Copy link
Copy Markdown
Member

@jhadvig jhadvig commented Feb 9, 2026

Problem
After enabling console plugins (particularly odf-console), the Console Operator gets stuck in a reconciliation loop with repeated console-config ConfigMap updates, causing console pods to restart every few minutes.

Root Cause
Non-deterministic YAML generation in the ConfigMap. Plugin-related data structures weren't consistently ordered, causing the ConfigMap content to vary between reconciliation cycles even when no actual changes occurred.

Solution
Fixed plugin data ordering:

  • Sort i18nNamespaces and proxyServices slices for deterministic output
  • Sort availablePlugins by name before processing

Added retry logic:
Wrap ApplyConfigMap() with retry.RetryOnConflict() to handle concurrent updates

/assign @TheRealJon

Summary by CodeRabbit

  • Bug Fixes

    • Improved ConfigMap update resilience with automatic retry on conflicts.
    • Fixed non-deterministic ordering of plugins and i18n namespaces to ensure consistent, predictable behavior.
    • Enhanced proxy service processing reliability through deterministic sorting.
  • Tests

    • Added test coverage for plugin sorting and i18n namespace ordering verification.

@openshift-ci-robot openshift-ci-robot added the jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. label Feb 9, 2026
@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Feb 9, 2026
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

@jhadvig: This pull request references Jira Issue OCPBUGS-74872, which is invalid:

  • expected the bug to target the "4.22.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Problem
After enabling console plugins (particularly odf-console), the Console Operator gets stuck in a reconciliation loop with repeated console-config ConfigMap updates, causing console pods to restart every few minutes.

Root Cause
Non-deterministic YAML generation in the ConfigMap. Plugin-related data structures weren't consistently ordered, causing the ConfigMap content to vary between reconciliation cycles even when no actual changes occurred.

Solution
Fixed plugin data ordering:

  • Sort i18nNamespaces and proxyServices slices for deterministic output
  • Sort availablePlugins by name before processing

Added retry logic:
Wrap ApplyConfigMap() with retry.RetryOnConflict() to handle concurrent updates

/assign @TheRealJon

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci Bot requested review from TheRealJon and spadgett February 9, 2026 15:29
@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 9, 2026
@jhadvig
Copy link
Copy Markdown
Member Author

jhadvig commented Feb 9, 2026

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Feb 9, 2026
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

@jhadvig: This pull request references Jira Issue OCPBUGS-74872, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.22.0) matches configured target version for branch (4.22.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @yapei

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci Bot requested a review from yapei February 9, 2026 15:33
Copy link
Copy Markdown
Member

@TheRealJon TheRealJon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Feb 9, 2026
@jhadvig
Copy link
Copy Markdown
Member Author

jhadvig commented Feb 10, 2026

/retest

Copy link
Copy Markdown
Member

@TheRealJon TheRealJon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Feb 17, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jhadvig, TheRealJon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jhadvig jhadvig added the acknowledge-critical-fixes-only Indicates if the issuer of the label is OK with the policy. label Mar 4, 2026
@yapei
Copy link
Copy Markdown
Contributor

yapei commented Mar 5, 2026

/assign @yanpzhan
for verification

@yanpzhan
Copy link
Copy Markdown

yanpzhan commented Mar 6, 2026

On cluster with the pr code, installed several dynamic plugins, console pods were running normally without restarting repeatedly, checked cm console-config in openshift-console namespace, here is the plugin list info:

    **i18nNamespaces:**
    - plugin__acm
    - plugin__console-demo-plugin
    - plugin__mce
    - plugin__monitoring-plugin
    - plugin__networking-console-plugin
    kind: ConsoleConfig
    monitoringInfo:
      alertmanagerTenancyHost: alertmanager-main.openshift-monitoring.svc:9092
      alertmanagerUserWorkloadHost: alertmanager-main.openshift-monitoring.svc:9094
    **plugins**:
      acm: https://console-chart-console-v2.open-cluster-management.svc.cluster.local:3000/plugin/
      console-demo-plugin: https://console-demo-plugin.console-demo-plugin.svc.cluster.local:9001/
      example: https://my-backend-service.default.svc.cluster.local:443/
      mce: https://console-mce-console.multicluster-engine.svc.cluster.local:3000/plugin/
      monitoring-plugin: https://monitoring-plugin.openshift-monitoring.svc.cluster.local:9443/
      networking-console-plugin: https://networking-console-plugin.openshift-network-console.svc.cluster.local:9443/
    **pluginsOrder:**
    - acm
    - mce
    - monitoring-plugin
    - networking-console-plugin
    - example
    - console-demo-plugin

@jhadvig in console-config.yaml, the 'i18nNamespaces' and 'plugins' list are sorted alphabetically, 'pluginsOrder' part is not sorted, need it to be sorted?

@openshift-bot
Copy link
Copy Markdown
Contributor

/jira refresh

The requirements for Jira bugs have changed (Jira issues linked to PRs on main branch need to target different OCP), recalculating validity.

@openshift-ci-robot
Copy link
Copy Markdown
Contributor

@openshift-bot: This pull request references Jira Issue OCPBUGS-74872, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.22.0) matches configured target version for branch (4.22.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @yapei

Details

In response to this:

/jira refresh

The requirements for Jira bugs have changed (Jira issues linked to PRs on main branch need to target different OCP), recalculating validity.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Apr 13, 2026

@openshift-ci-robot: GitHub didn't allow me to request PR reviews from the following users: yapei.

Note that only openshift members and repo collaborators can review this PR, and authors cannot review their own PRs.

Details

In response to this:

@openshift-bot: This pull request references Jira Issue OCPBUGS-74872, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.22.0) matches configured target version for branch (4.22.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @yapei

In response to this:

/jira refresh

The requirements for Jira bugs have changed (Jira issues linked to PRs on main branch need to target different OCP), recalculating validity.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci openshift-ci Bot removed the lgtm Indicates that a PR is ready to be merged. label May 27, 2026
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 27, 2026

New changes are detected. LGTM label has been removed.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 27, 2026

Note

.coderabbit.yaml has unrecognized properties

CodeRabbit is using all valid settings from your configuration. Unrecognized properties (listed below) have been ignored and may indicate typos or deprecated fields that can be removed.

⚠️ Parsing warnings (1)
Validation error: Unrecognized keys: "version", "path_filters"
⚙️ Configuration instructions
  • Please see the configuration documentation for more information.
  • You can also validate your configuration using the online YAML validator.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 80c84206-56b9-47dd-b94b-c3b68ae07816

📥 Commits

Reviewing files that changed from the base of the PR and between c54e288 and ecc4294.

📒 Files selected for processing (3)
  • pkg/console/operator/sync_v400.go
  • pkg/console/subresource/configmap/configmap.go
  • pkg/console/subresource/configmap/configmap_test.go

Walkthrough

This PR improves reconciliation stability by adding conflict-aware retry logic for ConfigMap updates and ensuring deterministic ordering across plugin processing and ConfigMap builder outputs. Changes span sync orchestration and configmap generation with corresponding test validation.

Changes

Deterministic Plugin Configuration and Resilient ConfigMap Sync

Layer / File(s) Summary
ConfigMap apply with conflict retry
pkg/console/operator/sync_v400.go
Adds sort and retry package imports, wraps resourceapply.ApplyConfigMap in retry.RetryOnConflict to retry on update conflicts instead of failing immediately.
Deterministic plugin processing in sync
pkg/console/operator/sync_v400.go
Sorts availablePlugins slice by plugin Name to ensure deterministic reconciliation order.
Deterministic ConfigMap builders
pkg/console/subresource/configmap/configmap.go
Sorts i18n namespace strings and proxy services by ConsoleAPIPath before returning, with added comments explaining determinism sources.
Sorting validation tests
pkg/console/subresource/configmap/configmap_test.go
Reorders availablePlugins in existing test scenario, adds TestPluginsWithI18nNamespaceSorting and TestGetPluginsProxyServicesSorting to validate sorted outputs with non-matching plugin types and empty input handling.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 14.29% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ❓ Inconclusive The description covers problem, root cause, and solution but is missing several required template sections like Test setup, Test cases, and Browser conformance. Add Test setup, Test cases, and Browser conformance sections to fully align with the repository's PR description template.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately reflects the main changes: sorting plugin lists to ensure deterministic behavior across reconciliation cycles.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

Review ran into problems

🔥 Problems

Git: Failed to clone repository. Please run the @coderabbitai full review command to re-trigger a full review. If the issue persists, set path_filters to include or exclude specific files.


Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci-robot openshift-ci-robot added jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. and removed jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. labels May 27, 2026
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

@jhadvig: This pull request references Jira Issue OCPBUGS-74872, which is invalid:

  • expected the bug to target either version "5.0." or "openshift-5.0.", but it targets "4.22" instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

Problem
After enabling console plugins (particularly odf-console), the Console Operator gets stuck in a reconciliation loop with repeated console-config ConfigMap updates, causing console pods to restart every few minutes.

Root Cause
Non-deterministic YAML generation in the ConfigMap. Plugin-related data structures weren't consistently ordered, causing the ConfigMap content to vary between reconciliation cycles even when no actual changes occurred.

Solution
Fixed plugin data ordering:

  • Sort i18nNamespaces and proxyServices slices for deterministic output
  • Sort availablePlugins by name before processing

Added retry logic:
Wrap ApplyConfigMap() with retry.RetryOnConflict() to handle concurrent updates

/assign @TheRealJon

Summary by CodeRabbit

  • Bug Fixes

  • Improved ConfigMap update resilience with automatic retry on conflicts.

  • Fixed non-deterministic ordering of plugins and i18n namespaces to ensure consistent, predictable behavior.

  • Enhanced proxy service processing reliability through deterministic sorting.

  • Tests

  • Added test coverage for plugin sorting and i18n namespace ordering verification.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@jhadvig
Copy link
Copy Markdown
Member Author

jhadvig commented May 27, 2026

/jira refresh
/retest

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels May 27, 2026
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

@jhadvig: This pull request references Jira Issue OCPBUGS-74872, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

No GitHub users were found matching the public email listed for the QA contact in Jira (yapei@redhat.com), skipping review request.

Details

In response to this:

/jira refresh
/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 27, 2026

@jhadvig: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-console ecc4294 link true /test e2e-aws-console
ci/prow/e2e-azure-ovn-upgrade ecc4294 link true /test e2e-azure-ovn-upgrade

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

acknowledge-critical-fixes-only Indicates if the issuer of the label is OK with the policy. approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants