chore: remove backwards compatibility with 2.6.0#5415
Conversation
|
@sbackend123 I think that removing all migration code might be a bit too eager - can you make sure that the migration code covers the currently active bee node version still showing up in swarmscan? |
I agree, as there are still 2.6 nodes (not many, but still ~6%), it is early to release bee without compatibility. This code is intended to be temporary. It is a good effort, but I would suggest to hold on merging, for some time. |
I ran the previous bee version (v2.6.0) locally, let it sync to current block height on maiinet and build a real data directory. Then I started this build on the same data directory without clearing it. The node started successfully with no migration errors. I also ran earlier versions trying to sync with mainnet, but get error |
I suppose it does not error the migrations because there's nothing to do in the migrations in the current build. I am not 100% sure about the reasoning behind the necessity of those migrations. But we should definitely leave in place the migration code to support the migration path from the node version still running on the network (2.6.0 definitely). The older ones we can mark as nops. |
Those 2.6.0 nodes are on some |
|
@sbackend123 this needs a rebase |
acud
left a comment
There was a problem hiding this comment.
there's a whole chunk of changes that are in pkg/storer/migration/all_steps.go that is completely not touched but the code can probably also be removed. as for the individual migration steps which are not in use already - there's no need of keeping the files with empty functions that do nothing - those can be removed.
|
|
||
| func firstByteString(data []byte) string { | ||
| if len(data) == 0 { | ||
| return "none" |
| // step_02 was a migration step that migrated the cache to a new format. | ||
| // It is now a NOOP since all nodes have already run this migration, | ||
| // and new nodes start with an empty database. | ||
| func step_02(_ transaction.Storage) func() error { |
There was a problem hiding this comment.
if this is not used anywhere - why do we need to keep the file?
…2.6.0 # Conflicts: # pkg/p2p/libp2p/version_test.go # pkg/statestore/storeadapter/migration.go # pkg/storer/internal/reserve/olditems.go # pkg/storer/migration/refCntSize.go # pkg/storer/migration/step_02.go # pkg/storer/migration/step_04_test.go # pkg/storer/migration/step_05.go # pkg/storer/migration/step_05_test.go # pkg/storer/migration/step_06.go # pkg/storer/migration/step_06_test.go
Add test. Move db repair tool.
If I understand everything correctly, we can't remove it completely because of our migration mechanism. |
|
|
||
| func (a *Address) UnmarshalJSON(b []byte) error { | ||
| v := &addressJSON{} | ||
| err := json.Unmarshal(b, v) |
There was a problem hiding this comment.
Nice work @sbackend123.
I just have a concern here, what will happen for the 2.6.0 nodes here ? because I see you removed Underlay field from addressJSON, I know this is not needed anymore, but in the Unmarshal the node will have the Underlays filed empty.
Maybe we can keep it in addressJSON (just to for the Unmarshal) and we get that Underlay addr and set it to Underlays slice
Right or maybe I'm missing some parts ?
There was a problem hiding this comment.
This is main point, we remove compatibility with 2.6.0 so minimal supported version would be 2.7.0. So we force everybody update their versions
There was a problem hiding this comment.
Yes, I know.
I’m just thinking out loud, if I were a node operator on 2.6.0 or earlier, I might suddenly get disconnected without really understanding why. That could be a bit confusing.
But as @janos mentioned, those nodes are only around 6%, so the impact seems pretty small.
| 5: noop, | ||
| 6: noop, | ||
| 7: noop, | ||
| 8: noop, |
There was a problem hiding this comment.
like in the other migrations comment - the sequence and indexes of these migrations should be commented such that they aren't removed in the future
acud
left a comment
There was a problem hiding this comment.
LGTM but will need a squash + rebase after the upcoming release
…2.6.0 # Conflicts: # pkg/bzz/address.go # pkg/bzz/address_test.go # pkg/bzz/export_test.go # pkg/bzz/underlay.go # pkg/bzz/underlay_test.go # pkg/hive/hive.go # pkg/p2p/libp2p/internal/handshake/handshake.go # pkg/p2p/libp2p/libp2p.go # pkg/statestore/storeadapter/migration.go
|
@sbackend123 linter failing |
|
@sbackend123 can you update the PR description? |
|
|
||
| // For 0 or 2+ addresses, the custom list format with the prefix is used. | ||
| // The format is: [prefix_byte][varint_len_1][addr_1_bytes]... | ||
| // The format is: [varint_len_1][addr_1_bytes]... |
There was a problem hiding this comment.
missing the prefix byte here. see comment on the left
| } | ||
| // The result is returned as a single-element slice for a consistent return type. | ||
| return []multiaddr.Multiaddr{addr}, nil | ||
| return deserializeList(data) |
There was a problem hiding this comment.
this doesn't seem right. with this change, the underlay should always start with the prefix byte. the if block above should check whether the first byte is not a prefix byte and throw if that is the case, then just deserialize like in the current if statement below (happy path not nested).
also some input length validation is needed since the deserialize call can panic on a short buffer (pass just 0x99 as the underlay)
There was a problem hiding this comment.
Fixed: in case if array of underlays contains only 0x99 prefix, decerialize will not panic, it just returns prefix (behavior unchanged)
|
|
||
| observedUnderlays, err := bzz.DeserializeUnderlays(resp.Syn.ObservedUnderlay) | ||
| if err != nil { | ||
| s.logger.Debug("handshake invalid synack observed underlay payload", "payload_len", len(resp.Syn.ObservedUnderlay), "first_byte", firstByteString(resp.Syn.ObservedUnderlay), "payload_prefix", payloadPrefix(resp.Syn.ObservedUnderlay), "error", err) |
There was a problem hiding this comment.
lots of stuff in this log line... not sure if it will be just swallowed in the rest of the logs. maybe just better to instrument the error with a tiny bit of information of what went wrong (not all the fields here). the same for the other logline
| return fmt.Sprintf("0x%02x", data[0]) | ||
| } | ||
|
|
||
| func payloadPrefix(data []byte) string { |
There was a problem hiding this comment.
not sure i understand why this is needed... maybe some info about why and how to use this... also why is n=16?
| if len(data) < n { | ||
| n = len(data) | ||
| } | ||
| return fmt.Sprintf("%x", data[:n]) |
There was a problem hiding this comment.
you can just fmt.Sprintf("%x",data[:min(16,len(data))])
| if len(payload) == 0 { | ||
| return fmt.Errorf("%w: observed underlay (len=0): %w", ErrInvalidSyn, err) | ||
| } | ||
| return fmt.Errorf("%w: observed underlay (len=%d, first=0x%02x): %w", ErrInvalidSyn, len(payload), payload[0], err) |
There was a problem hiding this comment.
why do we need conditional error formatting? this appears to be needed just so you have a sneak peek into the first byte (not sure what is the added value of this in production). i would suggest to get rid of this helper and just stringify the payload in the error formatting (one format). the 0x99 ascii character is the trademark sign so you would be able to see it in the logs correctly (if ever needed)
Checklist
Description
Breaking changes
Removes Bee 2.6.0 backward-compatibility code and the legacy single-multiaddr underlay wire encoding. Minimum supported Bee version on the network is 2.7.0; nodes on 2.6.0 or earlier will fail to complete a handshake with this build.
Underlay wire format
The 0x99 list prefix (underlayListPrefix) is kept. The only supported encoding is now always the prefixed list format, including for a single address or an empty list:
[0x99][varint_len_1][addr_1_bytes][varint_len_2][addr_2_bytes]...
Changes in detail:
Removed 2.6.0 compatibility shims
Database migrations
Legacy migration step implementations are removed; version indices are retained as NOOPs so existing data directories can still migrate safely (ValidateVersions requires contiguous version numbers). Comments document why these slots must not be removed or renumbered.
Also removes unused legacy helpers (olditems, oldstampindex, obsolete storer migration step files, etc.).
Open API Spec Version Changes (if applicable)
Motivation and Context (Optional)
Related Issue (Optional)
#5340
Screenshots (if appropriate):
AI Disclosure