test: harden offline coverage, coverage gate, and nightly integration - #39
Open
robertoecf wants to merge 1 commit into
Open
test: harden offline coverage, coverage gate, and nightly integration#39robertoecf wants to merge 1 commit into
robertoecf wants to merge 1 commit into
Conversation
Close reliability gaps for previously thin sources and production paths (429 retries, rate limits, CLI smoke) while keeping live API checks on a scheduled workflow instead of default PR CI. Co-authored-by: Cursor <cursoragent@cursor.com>
📝 WalkthroughWalkthroughThe pull request expands mocked coverage for data sources, API routes, CLI commands, retries, authentication, and rate limiting. CI adds a 60% Python 3.12 coverage gate and a scheduled or manual integration workflow. ChangesTest hardening and CI
Estimated code review effort: 3 (Moderate) | ~25 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/integration.yml:
- Around line 30-33: Update the integration test workflow step running `pytest
-m integration` to preserve pytest’s nonzero exit status through the `tee`
pipeline by enabling `pipefail` or explicitly checking the pytest result.
Broaden the `Upload failure log` condition to also run when the job is canceled,
using the appropriate `failure()`/`cancelled()` handling alongside
`steps.integration-tests.outcome == 'failure'`.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: f3946044-d109-48dc-8c07-a99f996f0448
📒 Files selected for processing (18)
.github/workflows/ci.yml.github/workflows/integration.ymlCHANGELOG.mdCONTRIBUTING.mdpyproject.tomlscripts/git/guardrails.shtests/conftest.pytests/test_api.pytests/test_api_sources_smoke.pytests/test_auth.pytests/test_b3_quotes.pytests/test_bcb_ptax_focus.pytests/test_cli.pytests/test_cvm_companies_financials.pytests/test_http_client.pytests/test_ibge.pytests/test_limits.pytests/test_tesouro_bonds.py
💤 Files with no reviewable changes (2)
- tests/test_api.py
- tests/conftest.py
Comment on lines
+30
to
+33
| run: pytest -m integration -v 2>&1 | tee pytest-integration.log | ||
|
|
||
| - name: Upload failure log | ||
| if: steps.integration-tests.outcome == 'failure' |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== files =="
git ls-files | rg '(^|/)(integration\.yml|.*workflow.*\.ya?ml|README|.*\.md)$' || true
echo
echo "== workflow excerpt =="
if [ -f .github/workflows/integration.yml ]; then
nl -ba .github/workflows/integration.yml | sed -n '1,100p'
fi
echo
echo "== search for related workflows/pytest logs =="
rg -n "Run integration tests|Upload failure log|pytest-integration\.log|steps\.integration-tests\.outcome|failure\(\)|set -o pipefail|pytest -m integration" .github/workflows . || true
echo
echo "== deterministic pipeline/status probes =="
python3 - <<'PY'
import subprocess, shlex, tempfile, os
cases = [
("normal_exit", "echo ok | tee /tmp/pytest-integration.log"),
("pytest_exit_pipefail_missing", "false | true"),
("pytest_exit_pipefail_set", "set -o pipefail; false | true"),
]
original = os.environ.get("SHELL")
os.environ["SHELL"] = "/bin/bash"
for name, cmd in cases:
rc = subprocess.run(["/bin/bash", "-c", cmd], text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print(f"{name}: cmd={cmd!r} returncode={rc.returncode} stdout={rc.stdout.strip()!r} stderr={rc.stderr.strip()!r}")
os.environ["SHELL"] = original or ""
from pathlib import Path
if Path("/tmp/pytest-integration.log").exists():
print("tmp log contains:", Path("/tmp/pytest-integration.log").read_text().strip())
PY
echo
echo "== bash status docs excerpt from local bash if available =="
if command -v bash >/dev/null 2>&1; then
bash --version | head -1
bash -c 'false | true; echo pipe=$?'
bash -c 'set -o pipefail; false | true; echo pipefail=$?'
fiRepository: robertoecf/OpenFinData
Length of output: 694
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== workflow excerpt =="
if [ -f .github/workflows/integration.yml ]; then
sed -n '1,120p' .github/workflows/integration.yml
fi
echo
echo "== search for related workflows/pytest logs =="
rg -n "Run integration tests|Upload failure log|pytest-integration\.log|steps\.integration-tests\.outcome|failure\(\)|set -o pipefail|pytest -m integration" .github/workflows . || true
echo
echo "== deterministic pipeline/status probes =="
python3 - <<'PY'
import subprocess, os
cases = {
"normal_exit": "echo ok | tee /tmp/pytest-integration.log",
"pytest_exit_pipefail_missing": "false | true",
"pytest_exit_pipefail_set": "set -o pipefail; false | true",
}
for name, cmd in cases.items():
rc = subprocess.run(["/bin/bash", "-c", cmd], text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print(f"{name}: cmd={cmd!r} returncode={rc.returncode} stdout={rc.stdout.strip()!r} stderr={rc.stderr.strip()!r}")
PY
echo
echo "== bash status probes =="
bash -c 'false | true; echo pipe=$?'
bash -c 'set -o pipefail; false | true; echo pipefail=$?'Repository: robertoecf/OpenFinData
Length of output: 2263
Preserve pytest failures before uploading the failure log.
The run command exits with tee’s status, so a failed pytest can make the step succeed without pipefail. Add Failure() or check() to the upload condition when the job can be canceled, because steps.integration-tests.outcome == 'failure' can have no value if the job is canceled.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/integration.yml around lines 30 - 33, Update the
integration test workflow step running `pytest -m integration` to preserve
pytest’s nonzero exit status through the `tee` pipeline by enabling `pipefail`
or explicitly checking the pytest result. Broaden the `Upload failure log`
condition to also run when the job is canceled, using the appropriate
`failure()`/`cancelled()` handling alongside `steps.integration-tests.outcome ==
'failure'`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
--cov-fail-under=60) and a scheduled nightlypytest -m integrationworkflow (plusworkflow_dispatch).Test plan
ruff format --check,ruff check,mypy src/findata,pytest tests/ -q→ 340 passed, 16 integration deselectedpytest --cov=findata --cov-fail-under=60→ ~68%Made with Cursor
Summary by CodeRabbit
Tests
Documentation
Chores