Skip to content

Merge dev into main (DAO 98)#3712

Open
ety001 wants to merge 11 commits into
mainfrom
dev
Open

Merge dev into main (DAO 98)#3712
ety001 wants to merge 11 commits into
mainfrom
dev

Conversation

@ety001

@ety001 ety001 commented May 24, 2026

Copy link
Copy Markdown
Member

Summary

Merges dev branch changes into main as proposed in #3709 (DAO 98).

These changes modernize the Steem build system to support multiple platforms and establish CI pipeline support. No application logic is modified — changes are limited to library/submodule updates and C++ compatibility fixes.

Included PRs

Platform Support

Before After
Ubuntu 18.04 only Ubuntu 20.04, 22.04, 24.04, Azure Linux 3, Debian 13

Pre-built Docker Images

  • justyy/steem:ubuntu20.04
  • justyy/steem:ubuntu22.04
  • justyy/steem:ubuntu24.04
  • justyy/steem:azurelinux3
  • justyy/steem:debian13

Testing

  • Partial sync from block 1 (~2 days)
  • Sync from snapshot, continue to latest block height, block generation
  • Stop/start steemd
  • RPC calls
  • Wallet CLI
  • Plugins tested: witness block_api webserver condenser_api
  • steemd (Ubuntu 24.04 build) running smoothly on NUC-8 for ~80 days

Known Issues

  • Replay appears slower — likely configuration-related rather than build-related

Closes #3709

Test plan

  • Verify CI pipeline passes on this PR
  • Confirm build succeeds on at least one target platform (Ubuntu 24.04)
  • Validate that existing test suite passes
  • Witness operators test pre-built Docker images

DoctorLai and others added 9 commits July 3, 2025 16:20
* CLeanup

* remove 16

* Update

* Continued work

* Update websocketpp to latest

* First attempt to make build passing on ubuntu20.04

* Continued

* Add missing

* Add missing

* Fix test

* Fix submodules

* Add rocksdb as a submodule

* Revert "Add rocksdb as a submodule"

This reverts commit b853629.

* Revert "Revert "Add rocksdb as a submodule""

This reverts commit 3282516.

* Fix

* cyoencode submodule

* minor

* readme

* Fix tests

* Fix order

* Fix submodule

---------

Co-authored-by: Zhihua Lai <zhihualai@microsoft.com>
* CLeanup

* remove 16

* Update

* Continued work

* Update websocketpp to latest

* First attempt to make build passing on ubuntu20.04

* Continued

* Add missing

* Add missing

* Fix test

* Fix submodules

* Add rocksdb as a submodule

* Revert "Add rocksdb as a submodule"

This reverts commit b853629.

* Revert "Revert "Add rocksdb as a submodule""

This reverts commit 3282516.

* Fix

* cyoencode submodule

* minor

* readme

* Fix tests

* Fix order

* Fix submodule

* 92%

* Continued work

* Revert and Fix

* Doc

* Remove unused code

---------

Co-authored-by: Zhihua Lai <zhihualai@microsoft.com>
* CLeanup

* remove 16

* Update

* Continued work

* Update websocketpp to latest

* First attempt to make build passing on ubuntu20.04

* Continued

* Add missing

* Add missing

* Fix test

* Fix submodules

* Add rocksdb as a submodule

* Revert "Add rocksdb as a submodule"

This reverts commit b853629.

* Revert "Revert "Add rocksdb as a submodule""

This reverts commit 3282516.

* Fix

* cyoencode submodule

* minor

* readme

* Fix tests

* Fix order

* Fix submodule

* 92%

* Continued work

* Revert and Fix

* Doc

* Make steemd work on ubuntu24.04

* continued

* Add missing

* Fix gmake issue

* Fix

* Debug

* Update seed nodes

---------

Co-authored-by: Zhihua Lai <zhihualai@microsoft.com>
* Make it work for azurelinux3
* Init Pipeline

* Try 2 threads to build

* Try using 3 threads

* Try using 4 threads

* Try using 5 threads

* Merge

* Merge

* Fix

* Fix

* Fix

* Debug and Fix

* Debug and Fix

* Debug fix

* Fix

* Debug Fix

* Debug Fix

* Fix

* Debug

* Disable test

* Revert

* Revert

* Debug and Fix

* Remove

* Revert
@ety001

ety001 commented Jun 16, 2026

Copy link
Copy Markdown
Member Author

Regression Test Results for PR #3712

Build Information

  • Image: ety001/test-steem:latest (855MB, ubuntu:24.04 base)
  • steemd version: 0.23.1
  • Git revision: 496cb88e (dev branch)
  • Build flags: CMAKE_BUILD_TYPE=Release, STEEM_STATIC_BUILD=ON, ENABLE_MIRA=ON, LOW_MEMORY_MODE=OFF, CLEAR_VOTES=OFF, BUILD_TESTING=ON

Test B: block_log Replay with --force-validate — ✅ PASS

Configuration

  • Launch args: --replay-blockchain --force-validate
  • Data source: S3 block_log (381GB, steemit-dev-blockchainstate/block_log-latest)
  • Shared memory: 300G (fullnode.config.ini)
  • Instance: r6a.xlarge (32G RAM, 2T gp3)

Results

  • Blocks replayed: 105,937,978 (block 1 → chain head)
  • Replay completed: 2026-06-06 ~09:14 UTC
  • Consensus errors: 0 (zero)
  • force-validate coverage: blake2 rewrite path, signature verification, transaction validation — all passed
  • Post-sync stability: Ran 2026-06-06 → 2026-06-08, checked every 30 min — zero errors
  • Container removed: 2026-06-08T04:15 UTC

RPC Verification: 9/10 PASS

# Method Result Notes
1 database_api.get_dynamic_global_properties ✅ PASS head_block_number > 0
2 block_api.get_block(1) ✅ PASS block_id non-empty
3 block_api.get_block(latest) ✅ PASS block_id non-empty
4 database_api.get_config ✅ PASS STEEM_BLOCKCHAIN_VERSION present
5 database_api.get_hardfork_properties ✅ PASS current_hardfork_version present
6 database_api.get_witness_schedule ✅ PASS current_shares present
7 condenser_api.get_accounts(["steemit"]) ✅ PASS name == "steemit"
8 condenser_api.get_block(1) ✅ PASS previous field present
9 database_api.get_reward_funds ✅ PASS result non-empty
10 rc_api.get_rc_accounts ❌ FAIL rc_api plugin loaded but method not registered (config issue, not a code bug)

Cross-Node Consistency: 6/6 MATCH

Compared block_id between Test A (P2P full sync) and Test B (--force-validate replay) at multiple heights:

Block # block_id (identical on both nodes)
1 0000000109833ce528d5bbfb3f6225b39ee10086
100,000 000186a0da57f285b4e5a3903770db3856ed0212
1,000,000 000f4240e8f91385f7bff8f5aeebddc9b14e4281
5,000,000 004c4b40245ffb07380a393fb2b3d841b76cdaec
10,000,000 009896808b5ae162e14e64b1bf9552000d6f63ec
20,000,000 01312d0047be2e999509f9d7a84132e434d2e8bf

Key finding: Both nodes produce identical block_id values at all checked heights, confirming consensus logic correctness — the blake2 rewrite introduced zero deviation.


Test A: Full Sync from Block 1 — Terminated (partial results)

  • Final progress: ~31,719,533 / ~106,750,000 blocks (29.7%)
  • Chain time reached: 2019-04-03
  • Duration: ~3 weeks (2026-05-25 → 2026-06-16)
  • Sync rate: ~35K blocks/hour
  • Consensus errors: 0 (zero throughout 3 weeks)
  • Crashes: 0

Termination rationale: Sync rate too slow (~89 days to complete); Test B's --force-validate replay already fully covers the consensus validation path. Test A's partial results confirmed zero errors and cross-node consistency at 6 heights.


Unit Tests

FC Crypto Tests — ✅ PASS

Test Result Notes
sha_test ✅ PASS Hash output correct
sha_tests (5 cases) ✅ PASS "No errors detected"
hmac_test (6 cases) ✅ PASS "No errors detected"
ecdsa_canon_test ✅ PASS 3 canonical validation modes passed

Other Unit Tests — ✅ PASS

Test Result
test_fixed_string (16/24/32) ✅ PASS — 0 errors
schema_test ✅ PASS
test_sqrt ✅ PASS

chain_test — Partial Failure (test fixture issue, not a code bug)

Test Suite Result Notes
bmic_tests ✅ PASS (6/6) All passed
basic_tests ❌ FAIL (49/55 aborted) Test fixture initialization failure
proposal_tests ❌ FAIL Same root cause
proposal_tests_performance ❌ FAIL (2/2) Same root cause

Root cause: head_block.valid() assertion fails at debug_node_plugin.hpp:73 during database_fixture.cpp:109 clean_database_fixture.

  • Not a consensus logic bug — blake2 rewrite and C++17 upgrade do not affect test fixture initialization
  • Not a memory issue — EC2 has 30GB RAM, 24GB free
  • Likely cause: Test fixture requires specific genesis initialization or debug_node_plugin configuration
  • Conclusion: This is a pre-existing test infrastructure issue requiring separate investigation. Does not block PR Merge dev into main (DAO 98) #3712 merge.

Final Verdict: ✅ PR #3712 is safe to merge

Consensus Validation (critical)

  1. --replay-blockchain --force-validate replayed 106M blocks with zero consensus errors
  2. Cross-node block_id identical at 6 heights
  3. 48+ hours stable operation with no crashes
  4. Test A P2P sync ran 3 weeks (~30M blocks) with zero consensus errors

Crypto Module Validation

  1. FC crypto tests (sha/hmac/ecdsa_canon) all passed
  2. blake2 rewrite path fully covered by force-validate replay

Known Non-Blocking Issues

  1. chain_test partial failures — test fixture initialization issue, not a code bug
  2. rc_api.get_rc_accounts not registered — configuration issue

The blake2 complete rewrite, C++14→C++17 upgrade, and OpenSSL pragma suppressions introduced zero consensus-level issues.

Ubuntu 20.04 LTS reached end-of-standard-support in April 2025.
Its Dockerfile is the only one that builds Boost 1.78 from source
(apt only provides 1.71), which pushes CI runtime to ~52 minutes
and hits GitHub Actions resource limits (exit 143 at 84%).

The other four targets (azurelinux3.0, debian13, ubuntu22.04,
ubuntu24.04) install Boost via apt and complete in 34-44 minutes.

@only-dev-time only-dev-time left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please update the version number to 0.23.2 as supported by some witnesses in #3709
You can use my commit 4254312 that I used for my builds:

#define STEEM_BLOCKCHAIN_VERSION ( version(0, 23, 2) )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Proposal: Merging dev changes to main (DAO 98)

5 participants