Upstream the script thats used to generate patch for AST2700#568
Upstream the script thats used to generate patch for AST2700#568chander-nexthop wants to merge 5 commits into
Conversation
… 2700 2. Script to generate the patch that adds support for Aspeed AST2700 SOC. The script will generate the patch in WORK_DIR/aspeed-ast2700-support-new.patch This needs to be moved to patches-sonic/aspeed-ast2700-support.patch. Signed-off-by: Chandrasekaran Swaminathan <chander@nexthop.ai>
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
Thank you for sharing the script. Please make this two merge/pull requests. |
Signed-off-by: Chandrasekaran Swaminathan <chander@nexthop.ai>
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Done. |
| else | ||
| echo "Warning: Patch file not found: $line" | ||
| fi | ||
| done < <(head -228 "$KERNEL_DIR/patches-sonic/series") |
There was a problem hiding this comment.
Why the first 228 lines specifically?
There was a problem hiding this comment.
Thank you @saiarcot895 for pointing it out . It a hardcoded cutoff for the pre aspeed section of the series file . it is replaced in the latest commit with an aspeed section skip so it adapts as patches are added or reordered.
| cat > "$WORK_DIR/smart_merge.py" << 'PYTHON_EOF' | ||
| #!/usr/bin/env python3 | ||
| """ | ||
| Smart merge for Kconfig/Makefile files. | ||
| Uses a common ancestor approach: extract Aspeed additions from diff, | ||
| add them to SONiC base at correct locations while preserving order. | ||
| """ | ||
| import sys | ||
| import re | ||
| import difflib | ||
|
|
||
| def smart_merge(sonic_file, aspeed_file, target_file): | ||
| # Aspeed-related patterns | ||
| aspeed_patterns = [ | ||
| re.compile(r'aspeed', re.IGNORECASE), | ||
| re.compile(r'ast2[567]00', re.IGNORECASE), | ||
| re.compile(r'ast1[78]00', re.IGNORECASE), | ||
| re.compile(r'AST2700_IRQ'), | ||
| re.compile(r'ARCH_ASPEED'), | ||
| ] | ||
|
|
||
| def is_aspeed_related(text): | ||
| return any(pattern.search(text) for pattern in aspeed_patterns) | ||
|
|
||
| # Read files | ||
| with open(sonic_file, 'r') as f: | ||
| sonic_lines = f.readlines() | ||
|
|
||
| with open(aspeed_file, 'r') as f: | ||
| aspeed_lines = f.readlines() | ||
|
|
||
| # Use difflib to get a proper sequence matcher | ||
| matcher = difflib.SequenceMatcher(None, sonic_lines, aspeed_lines) | ||
|
|
||
| result = [] | ||
|
|
||
| for tag, i1, i2, j1, j2 in matcher.get_opcodes(): | ||
| if tag == 'equal': | ||
| # Common lines - keep them | ||
| result.extend(sonic_lines[i1:i2]) | ||
| elif tag == 'delete': | ||
| # Lines deleted in aspeed - KEEP THEM (preserve SONiC content) | ||
| result.extend(sonic_lines[i1:i2]) | ||
| elif tag == 'insert': | ||
| # Lines added in aspeed - check if Aspeed-related | ||
| new_lines = aspeed_lines[j1:j2] | ||
| aspeed_related = any(is_aspeed_related(line) for line in new_lines) | ||
| if aspeed_related: | ||
| # Add these lines | ||
| result.extend(new_lines) | ||
| # Otherwise skip (non-Aspeed additions) | ||
| elif tag == 'replace': | ||
| # Lines changed - keep SONiC version, then add Aspeed additions if any | ||
| result.extend(sonic_lines[i1:i2]) | ||
| new_lines = aspeed_lines[j1:j2] | ||
| aspeed_related = any(is_aspeed_related(line) for line in new_lines) | ||
| if aspeed_related: | ||
| # Also add the Aspeed replacements | ||
| result.extend(new_lines) | ||
|
|
||
| # Write result | ||
| with open(target_file, 'w') as f: | ||
| f.writelines(result) | ||
|
|
||
| if __name__ == '__main__': | ||
| smart_merge(sys.argv[1], sys.argv[2], sys.argv[3]) | ||
| PYTHON_EOF |
There was a problem hiding this comment.
Suggestion for the future: split this into a separate file just for editing/review purposes.
There was a problem hiding this comment.
it is taken care @saiarcot895 . extracted into scripts/smart_merge.py now. Thank you.
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
paulmenzel
left a comment
There was a problem hiding this comment.
700 lines is quite much. Why can’t nexthop not publish a git repository with the all the SONiC patches applied, and the Aspeed patches on top? (SONiC should also publish it’s own Linux kernel git archive with the patches applied.)
| # Example, run with ASPEED_TAG=v00.07.02 ./scripts/generate-aspeed-patch.sh | ||
| ASPEED_BRANCH="${ASPEED_TAG:-aspeed-master-v6.12}" | ||
| ASPEED_REPO="https://github.com/AspeedTech-BMC/linux.git" | ||
| WORK_DIR="/tmp/aspeed-patch-gen" |
There was a problem hiding this comment.
Please consider $TMPDIR.
There was a problem hiding this comment.
Thank you @paulmenzel for the feedback. review comment incorporated .
| OUTPUT_PATCH="$WORK_DIR/aspeed-ast2700-support-new.patch" | ||
|
|
||
| # Read kernel version from Makefile | ||
| KERNEL_VERSION=$(grep "^KERNEL_VERSION" "$KERNEL_DIR/Makefile" | cut -d= -f2 | tr -d ' ') |
There was a problem hiding this comment.
sed -n 's/^KERNEL_VERSION\s*=\s*//p' "$KERNEL_DIR/Makefile"
There was a problem hiding this comment.
KERNEL_VERSION ?= 6.12.41
A literal = regex doesnt match ?=, so the variable would expand to empty and break the rest of the script. I generalized the pattern. Thank you for feedback .
| fi | ||
| mkdir -p "$SONIC_SRC" | ||
| cd "$SONIC_SRC" | ||
| wget -q "$SONIC_KERNEL_URL" |
There was a problem hiding this comment.
Why redownload it each time?
There was a problem hiding this comment.
Tarball is now cached . Thank you.
| wget -q "$SONIC_KERNEL_URL" | ||
| tar -xf "linux_${KERNEL_VERSION}.orig.tar.xz" --strip-components=1 | ||
| rm "linux_${KERNEL_VERSION}.orig.tar.xz" | ||
| echo "SONiC kernel source ready: $(du -sh $SONIC_SRC | cut -f1)" |
There was a problem hiding this comment.
Explicitly mention the size?
There was a problem hiding this comment.
Explicitly mentioned the size now. Thank you.
- Updated patch series file to move ASPEED patches at the end - Move inline smart_merge Python out of the bash heredoc into scripts/smart_merge.py; bash invokes it via $SMART_MERGE. - Replace the hard-coded "head -228" cap with a break at the "###-> aspeed" marker so the patch-application loop stays correct as patches-sonic/series evolves. - Warn patches listed after "###-> aspeed-end" with a note that they may need regeneration if they touch files also modified by the regenerated aspeed patch. Signed-off-by: Chinmoy Dey <chinmoy@nexthop.ai>
df0c469 to
5e82787
Compare
|
/azp run |
|
Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command. |
- WORK_DIR now honors ${TMPDIR:-/tmp}.
- KERNEL_VERSION read via sed (handles ?= / := / += / =).
- SONiC kernel tarball is cached across runs; only the extracted
$SONIC_SRC tree is wiped and re-extracted each time.
- progress lines now label the "size on disk"
figure explicitly.
Merge remote-tracking branch 'upstream/master' into chander-ast2700-patch-script
Signed-off-by: Chinmoy Dey <chinmoy@nexthop.ai>
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
All review comments have been incorporated. |
|
@saiarcot895 @paulmenzel , please let me know if you have any additional feedback on this PR. Otherwise, we can proceed with the next steps here. |
|
|
||
| ############################################################ | ||
| # | ||
| # Let us keep Aspeed patches at the end |
There was a problem hiding this comment.
Is there a reason for keeping Aspeed patches at the end?
There was a problem hiding this comment.
The Aspeed generated patch block is intentionally kept at the end to ensure that the base patch set is applied successfully before any Aspeed specific patches are processed. This avoids patch ordering issues and keeps the script behaviour deterministic as patches-sonic/series changes over time. Any new patch added to the patch stack should be validated to ensure it does not introduce conflicts with the Aspeed generated patches.
| 0003-neighbor-Add-NTF_EXT_VALIDATED-flag-for-externally-v.patch | ||
|
|
||
| ###-> nvidia_aspeed_bmc-start | ||
| ###-> nvidia_aspeed_bmc-end |
There was a problem hiding this comment.
Nvidia's aspeed patch marker ordering has been swapped, which may cause impact to them.
There was a problem hiding this comment.
This is just a marker at the moment, and there are no patches in this section yet. If its current placement is a concern, I can certainly move it to the end as well.
The main intent is to provide a clear location for any future Aspeed generated patches. Whenever a new patch is introduced that touches the same files as the Aspeed generated patches, we should ensure that any conflicts or interactions are reviewed and resolved appropriately. That is the primary expectation behind keeping this marker at the end.
I haven’t seen a reply to this comment. |
Thats a fair question. The main reason for upstreaming the script is reproducibility. Rather than maintaining a separate rebased kernel tree, the script regenerates the AST2700 patch from the current SONiC kernel source and the selected Aspeed baseline. The recent changes were primarily focused on making the workflow more maintainable and less ad hoc. The goal of this PR is simply to upstream the generation method so that others can reliably reproduce the same patch set. |
As discussed in the SONiC-BMC work group meeting yesterday I am submitting the script that I use to generate the patch file to add support for AST 2700 to the SONiC Linux Kernel. The script by default will create the patch based on the current top-of-tree in aspeed-master-v6.12. This can be overridded by running the script with the env var ASPEED_TAG, ex "ASPEED_TAG=v00.07.02 ./scripts/generate-aspeed-patch.sh"