Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
152 commits
Select commit Hold shift + click to select a range
1fbb68d
initial project commit
rbazin May 6, 2025
3401fe6
fix: fixing earthplot figure to PIL image conversion
rbazin May 7, 2025
07309c1
feat: implementing secured private VLLM inference server
rbazin May 10, 2025
7b72577
doc: adding doc for VLLM inference server configuration and deployment
rbazin May 10, 2025
171df87
feat: adding nvidia docker runtime configuration
rbazin May 10, 2025
d88edb3
doc: small update about nvidia runtime
rbazin May 10, 2025
befdbda
doc: updating comment
rbazin May 10, 2025
e234494
feat: creating custom Dockerfile for vllm
rbazin May 15, 2025
06078fd
feat: adding hf hub token as env variable in docker file
rbazin May 15, 2025
9680663
fix: removing kwarg in vllm command line
rbazin May 15, 2025
e4044fc
feat: adding model download in setup script
rbazin May 16, 2025
fd326d8
Fixing Dockerfile for vllm server
rbazin May 16, 2025
54bf040
feat: adding model served name for model loaded locally
rbazin May 18, 2025
70f3d8f
feat: implementing multi stages prompt, with clear requirements break…
rbazin May 20, 2025
4750075
feat: interverting method and requirements in prompt to make method m…
rbazin May 20, 2025
ac48de8
feat: implementing function to generate special weather events images…
rbazin May 21, 2025
062474f
feat: switching caddy for traefik reverse proxy
rbazin May 24, 2025
b88b9ed
feat: replacing caddy with traefik reverse proxy
rbazin May 24, 2025
4b67502
feat: updating example and instructions files for reverse proxy
rbazin May 24, 2025
919dd3a
feat: adding todos for security against bots
rbazin May 24, 2025
bcd90d8
fix: fixing project import issue
rbazin Jun 1, 2025
fad246f
feat: implementing LLM wrapper class for openai and groq llm api prov…
rbazin Jun 1, 2025
346bd06
feat: enhancing user prompt with task decomposition and output format
rbazin Jun 1, 2025
ed1501e
feat: developing improved prompt
rbazin Jun 1, 2025
a5d1729
feat: creating cli for description generations
rbazin Jun 5, 2025
eaaf738
doc: commenting fire and ipython compatibility issue
rbazin Jun 5, 2025
d0cf58d
fix: fixing wrong groq url endpoint
rbazin Jun 6, 2025
561d595
feat: adding logging
rbazin Jun 6, 2025
2e92951
feat: adding earthkit plot figure as possible input to generator agent
rbazin Jun 10, 2025
a42ad90
feat: updating the user prompt with extracted figure metadata
rbazin Jun 10, 2025
eeab6f0
feat: creating orchestrator placeholder file
rbazin Jun 10, 2025
e91bae5
feat: fixing value error message for figure metadata extraction
rbazin Jun 10, 2025
1c6a9ea
doc: adding metadata hanling todo
rbazin Jun 10, 2025
6d0074a
refact: refactoring llm creation
rbazin Jun 23, 2025
db4fc43
feat: implementing single criteria evaluator agent
rbazin Jun 23, 2025
cd0852c
fix: fixing typos
rbazin Jun 23, 2025
b09f81c
refact: refactoring LLM creation
rbazin Jun 23, 2025
4c15dc6
feat: writing prompts for each quality criteria evaluation agent
rbazin Jun 23, 2025
e324cca
feat: implementing evaluator agent orchestrating criteria agents
rbazin Jun 24, 2025
92447c3
feat: removing criteria name from llm output to increase reliability
rbazin Jun 24, 2025
d1456e2
refact: refactoring prompt organization
rbazin Jun 24, 2025
b8393be
refact: improving todo item for clarity
rbazin Jun 27, 2025
1e10a9d
refact: renaming prompt file for clarity
rbazin Jun 27, 2025
3759255
refact: replacing criterion attribute by name in dataclass for clarity
rbazin Jun 27, 2025
c951142
feat: implementing generator method to append user prompt
rbazin Jun 27, 2025
65d9b8a
feat: implementing orchestrator class
rbazin Jun 27, 2025
6255b65
doc: improving docstring with return info
rbazin Jun 27, 2025
76284fd
refact: renaming template placeholder name for clarity
rbazin Jun 27, 2025
9ad969b
feat: implementing CLI for evaluator agent
rbazin Jul 1, 2025
4063114
feat: creating enum as registry for quality criteria
rbazin Jul 1, 2025
b3f1c62
feat: creating placeholder logic for limits acknowledgement
rbazin Jul 1, 2025
ff87c46
feat: implementing static acknowledgement of limits
rbazin Jul 1, 2025
09e3062
refact: removing deprecated main function
rbazin Jul 1, 2025
fcefda6
feat: using system registry to get valid criteria
rbazin Jul 4, 2025
f148bcd
fix: fixing evaluator parsing bug with wrong required fields
rbazin Jul 4, 2025
0143996
refact: renaming main script to cli
rbazin Jul 5, 2025
909335c
feat: adding orchestrator to CLI
rbazin Jul 5, 2025
8e609c8
doc: improving setup and usage readme instructions
rbazin Jul 5, 2025
f9814c3
doc: adding project structure section to readme
rbazin Jul 5, 2025
e81825f
feat: implement generator prompt improvements from labeled descriptio…
rbazin Jul 28, 2025
e7e99d4
feat: adding google gemini as supported LLM provider
rbazin Jul 28, 2025
2a044ef
fix: removing repeated step 6 in generator steps
rbazin Jul 28, 2025
cbbb4ec
feat: improving evaluator prompts according to expert labeled descrip…
rbazin Jul 28, 2025
3f4bb3c
feat: improving fluency prompt
rbazin Jul 29, 2025
8935b05
fix: fixing LLMInterface name error
rbazin Jul 29, 2025
64d7559
feat: improving type safety in cli
rbazin Jul 29, 2025
7b4bc01
feat: improving type safety in orchestrator
rbazin Jul 29, 2025
19a44e7
feat: implementing first version of pressure center extractor
rbazin Jul 29, 2025
a892ef0
feat: creating centralized logging config
rbazin Jul 31, 2025
732c1cd
fix: fixing typo
rbazin Jul 31, 2025
cee9b31
modifying data extractor interface
rbazin Jul 31, 2025
5544ac1
doc: adding todo to prepare data extraction from fieldlist object
rbazin Jul 31, 2025
154d6d7
fix: fixing typo in metadata prompt update from figure object
rbazin Jul 31, 2025
1b4f0d7
refact: renaming package as earth reach
rbazin Aug 2, 2025
687e069
feat: adding gemini and claude models as optional dependencies
rbazin Aug 2, 2025
4cee2ee
feat: creating earth reach agent template class for future main integ…
rbazin Aug 2, 2025
d372ebf
feat: adding pre-commit hooks with ruff configured to increase code q…
rbazin Aug 3, 2025
1ee83cf
feat: adding more ruff linting rules to increase code quality, adding…
rbazin Aug 3, 2025
d753e51
fix: fixing trailing whitespaces with pre-commit hooks
rbazin Aug 3, 2025
8c61b56
feat: creating dedicated extractors subpackage
rbazin Aug 3, 2025
3ea5e1d
fix: fixing ruff linting rules
rbazin Aug 3, 2025
6e48908
doc: updating todo items
rbazin Aug 3, 2025
3b4f417
fix: fixing base extractor import statement
rbazin Aug 3, 2025
eafb63b
feat: adding mypy configuration to increase code quality with type ch…
rbazin Aug 3, 2025
c793ae9
fix: fixing minor type checking issues
rbazin Aug 3, 2025
34c523a
feat: implementing unit test structure
rbazin Aug 3, 2025
b4fcfae
feat: improving type coherence for images
rbazin Aug 9, 2025
9e5da6c
feat: implementing simple local-extrema pressure center extractor
rbazin Aug 10, 2025
77cca19
feat: updating output string if not pressure centers detected
rbazin Aug 10, 2025
a667a13
feat: preprocessing average seasonal regional temperature data
rbazin Aug 12, 2025
6e03b99
feat: minor improvements
rbazin Aug 12, 2025
bedbf56
feat: implementing template for regional temperature feature extractor
rbazin Aug 12, 2025
d3bf386
chore: ignoring data file
rbazin Aug 18, 2025
0d5e0fc
feat: removing unused evaluation heuristics
rbazin Aug 18, 2025
4d9b481
doc: creating structure for sphinx doc
rbazin Aug 18, 2025
7c41b3a
doc: adding module-level docstring
rbazin Aug 18, 2025
3ffea27
doc: writing docstring for orchestrator
rbazin Aug 18, 2025
e26e390
feat: adding model name as env variable
rbazin Aug 18, 2025
9ab708d
feat: removing unused dotenv env loading
rbazin Aug 18, 2025
9fac17c
feat: adding automatic api doc build to makefile
rbazin Aug 18, 2025
6567bf4
doc: improving introduction paragraphs in index
rbazin Aug 18, 2025
4589331
doc: writing installation page
rbazin Aug 18, 2025
824ad28
fix: fixing source file path in docs
rbazin Aug 19, 2025
f5aa534
doc: writing installation doc
rbazin Aug 19, 2025
0366fc2
doc: writing draft of API reference
rbazin Aug 19, 2025
413e82f
fix: removing auto doc build from makefile
rbazin Aug 19, 2025
6fa0685
doc: writing CLI usage doc
rbazin Aug 19, 2025
95b1762
doc: writing section descriptions
rbazin Aug 19, 2025
564cd0a
doc: improving docstrings in source code
rbazin Aug 19, 2025
cde152b
doc: update module imports and exports in __init__.py
rbazin Aug 19, 2025
de54c19
feat: implementing quickplot main integration point
rbazin Aug 19, 2025
b790d6f
doc: adding raised error to orchestrator
rbazin Aug 19, 2025
699203a
feat: adding main components to package level import
rbazin Aug 19, 2025
70048b5
chore: upadting earthkit data import statement for consistency
rbazin Aug 19, 2025
53daa7d
doc: writing first version of library API usage
rbazin Aug 19, 2025
637fa97
feat: adding llm provider and model name as arguments to earthreachag…
rbazin Aug 19, 2025
dc5c412
doc: updating with provider information
rbazin Aug 19, 2025
faa7265
fix: fixing wrong spacing between bullet points
rbazin Aug 19, 2025
773b4e8
doc: clarifying CLI limit in doc
rbazin Aug 19, 2025
7482f89
doc: removing not working extensions
rbazin Aug 19, 2025
290d33b
feat: improving error handling with data extractors
rbazin Aug 19, 2025
f403216
feat: improving logging in orchestrator
rbazin Aug 19, 2025
5a9f230
fix: fixing logging level
rbazin Aug 19, 2025
40153f5
doc: using real model in example
rbazin Aug 19, 2025
5e8505d
feat: adding extracted data information to generator and evaluator pr…
rbazin Aug 19, 2025
8ea83f5
feat: improving formatted features string from pressure data extractor
rbazin Aug 19, 2025
49dd237
feat: removing logging in pressure extractor validation
rbazin Aug 19, 2025
efc4f46
feat: removing success validation in main agent
rbazin Aug 19, 2025
b5e16f9
doc: updating docstring
rbazin Aug 19, 2025
ab1e141
feat: removing dev notebook
rbazin Aug 19, 2025
fbd1132
feat: creating demo notebook
rbazin Aug 19, 2025
5aeef21
chore: improving comment phrasing
rbazin Aug 19, 2025
3f4ff89
fix: fixing wrong root dir path
rbazin Aug 19, 2025
dce4800
fix: fixing variable typo
rbazin Aug 19, 2025
20e8c8d
fix: verifying data is not none in orchestrator
rbazin Aug 19, 2025
71d49e5
doc: clarifying main entrypoint usage
rbazin Aug 19, 2025
6f924e3
doc: adding pointer to example notebook
rbazin Aug 19, 2025
3a39c4f
doc: adding pointer for CDS api key
rbazin Aug 19, 2025
c1f624b
doc: adding example of supported charts
rbazin Aug 19, 2025
e48fa19
doc: adding usage example
rbazin Aug 19, 2025
edae622
doc: updating readme vllm instructions
rbazin Aug 19, 2025
94e644f
chore: adding todo
rbazin Aug 19, 2025
c7d6c5d
doc: improving readme header
rbazin Aug 19, 2025
a50e7ee
doc: removing subsections
rbazin Aug 19, 2025
2605fac
ci: implementing code quality analysis CI
rbazin Aug 20, 2025
b804e41
fix: fixing typo in model name
rbazin Aug 20, 2025
43c66e7
feat: improving logging coverage at main decision points
rbazin Aug 26, 2025
e114a55
fix: removing todo items
rbazin Aug 31, 2025
32af211
feat: updating instructions and setup script with traefik specific in…
rbazin Aug 31, 2025
cf0008c
refact: removing temperature extractor from project
rbazin Aug 31, 2025
0f3647b
Merge branch 'main' into develop
rbazin Aug 31, 2025
bef0daf
Revert "Merge branch 'main' into develop"
rbazin Sep 1, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: CI

on:
push:
branches: [ main, develop ]
paths-ignore:
- '**.md'
- 'docs/**'
- 'README*'
- '.gitignore'
pull_request:
branches: [ main, develop ]
paths-ignore:
- '**.md'
- 'docs/**'
- 'README*'
- '.gitignore'
workflow_dispatch:

jobs:
code-analysis:
name: Code Quality Analysis
runs-on: ubuntu-latest

steps:
- name: Checkout Repository
uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v5
with:
version: "latest"

- name: Install Python
run: uv python install

- name: Install dependencies
run: uv sync --locked --all-extras --dev

- name: Check formatting
run: uv run ruff format --check .

- name: Check linting
run: uv run ruff check .
11 changes: 3 additions & 8 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,6 @@ wheels/
# Virtual environments
.venv

# Notebooks
.ipynb_checkpoints/
*.ipynb

# Data
weather_images/

Expand All @@ -22,7 +18,6 @@ weather_images/
# Common
*.DS_Store

# Images
*.png
*.jpg
*.jpeg
# Data
data/monthly_average.zarr
data/*.nc
18 changes: 18 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-files
- id: check-merge-conflict
- id: check-toml
- id: debug-statements

- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.12.7
hooks:
- id: ruff-check
args: [--fix]
- id: ruff-format
1 change: 1 addition & 0 deletions .python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.12
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -199,4 +199,4 @@
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
limitations under the License.
105 changes: 90 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# EarthReach Agent: Dual-LLM Framework for Validated Meteorological Chart Descriptions

<p align="center">
<a href="https://www.python.org/downloads/release/python-3120/">
<img src="https://img.shields.io/badge/python-3.12-blue.svg" alt="Python 3.12">
</a>
<a href="https://opensource.org/licenses/apache-2-0">
<img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License: Apache 2.0">
</a>
Expand All @@ -14,14 +17,19 @@

## EarthReach

EarthReach is a challenge from the 2025 edition dedicated to enhancing the accessibility of meteorological data visualisations produced by Earthkit, by equipping the plots module with LLM-powered alternative text generation capabilities.
EarthReach is a Python library for generating natural language descriptions of meteorological data visualizations. The library extends [earthkit-plots](https://github.com/ecmwf/earthkit-plots) by providing automated text generation capabilities for weather charts, enabling programmatic conversion of visual data representations into structured textual descriptions.

The system implements a dual-LLM architecture consisting of a generator agent and an evaluator agent. The generator creates initial descriptions from chart images and associated GRIB file metadata, while the evaluator assesses output quality across multiple criteria including scientific accuracy, coherence, and meteorological relevance. This iterative process continues until quality thresholds are met or maximum iterations are reached.

![Global temperature and mean sea level map](./docs/source/_static/example_global_chart.png)

## Installation

### Prerequisites

- Python 3.12 or higher
- [uv](https://docs.astral.sh/uv/) - Python package and project manager
- [uv](https://docs.astral.sh/uv/): Python package and project manager (will automatically install Python 3.12+ if needed)
- [Climate Data Store ](https://cds.climate.copernicus.eu/how-to-api): API key configured for accessing meteorological data
- API key for a supported LLM provider (OpenAI, Google Gemini, Anthropic Claude, Groq, or any OpenAI-compatible API provider)

### Setup

Expand All @@ -32,42 +40,63 @@ EarthReach is a challenge from the 2025 edition dedicated to enhancing the acces
```
2. **Create a virtual environment and install dependencies**
```sh
uv sync
uv sync --group dev
```
This command will automatically:
- Create a .venv virtual environment
- Install all project dependencies from pyproject.toml
- Install development dependencies

3. **Activate the virtual environment**
```sh
source .venv/bin/activate # On Windows, use: .venv\Scripts\activate
```

4. **Set up pre-commit hooks (recommended for development)**
```sh
uv run pre-commit install
```

You're now ready to use the project.

## Project Structure

```sh
.
.
├── docs/ # Project documentation
├── notebooks/ # Tutorials & experiments
├── src/
│ ├── earth_reach_agent/ # Main package
│ ├── earth_reach/ # Main package
│ └── tests/ # Unit and integration tests (to come)
├── vllm/ # VLLM inference server setup
├── pyproject.toml # Project dependencies and metadata
└── uv.lock # Locked dependency versions
```

## VLLM Inference Server
## Basic Usage

To run this project, you will need to have an openAI-compatible LLM inference server.
```python
from earth_reach import EarthReachAgent
import earthkit.plots as ekp
import earthkit.data as ekd

We provide instructions on how to setup your own secured inference server using [VLLM](./vllm/setup.md).
# Load your data with earthkit-data
data = ekd.from_source("file", "your_data.grib")

## Usage
# Create a weather chart with earthkit-plots
figure = ekp.quickplot(data, mode="overlay")

# Generate description
agent = EarthReachAgent(provider="openai")
description = agent.generate_alt_description(figure, data)
print(description)
```

The project provides a command-line interface (CLI) accessible through `earth-reach-agent` or its shorter alias `era`.
See `notebooks/example.ipynb` for a practical usage example.

## CLI Interface

EarthReach includes a standalone CLI that works on image files only, producing less detailed descriptions than the full library integration.

### Available Commands

Expand All @@ -76,20 +105,66 @@ View all commands and options:
uv run era --help
```

### Generate weather chart descriptions

Generate a natural language description from a weather chart image:
```sh
uv run era generate --image-path <path_to_image>
```

### Evaluate descriptions

Evaluate the accuracy of a description against a weather chart:

```sh
uv run era evaluate --image-path <path_to_image> --description "<description_string>"
```
## VLLM Inference Server

EarthReach supports any OpenAI-compatible API endpoint for self-hosted LLMs. See `vllm/` directory for a VLLM setup example.

**Warning**: Self-hosting requires advanced system administration skills and significant GPU resources. Recommended only for experienced users.

## Development

### Development Tooling

This project uses the following development tools:

- **Ruff**: Fast Python linter and formatter
- **mypy**: Static type checker for Python
- **Pre-commit**: Git hooks for code quality checks
- **Pytest**: Testing framework

### Code Quality

The project is configured with pre-commit hooks that run automatically before each commit to ensure code quality:

- **Ruff linting**: Checks for common Python issues and enforces coding standards
- **Ruff formatting**: Automatically formats code for consistency
- **mypy type checking**: Validates type hints and catches import errors
- **Basic checks**: Trailing whitespace, file endings, YAML/TOML syntax, merge conflicts

### Running Tools Manually

You can run the development tools manually:

```sh
# Run ruff linter
uv run ruff check .

# Run ruff formatter
uv run ruff format .

# Run mypy type checker
uv run mypy .

# Run pre-commit hooks on all files
uv run pre-commit run --all-files
```

### Configuration

- Ruff configuration is in `pyproject.toml` under `[tool.ruff]`
- mypy configuration is in `pyproject.toml` under `[tool.mypy]`
- Pre-commit configuration is in `.pre-commit-config.yaml`

## License

See [LICENSE](LICENSE)
Expand Down
Empty file removed docs/.gitkeep
Empty file.
20 changes: 20 additions & 0 deletions docs/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Minimal makefile for Sphinx documentation
#

# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?=
SPHINXBUILD ?= sphinx-build
SOURCEDIR = source
BUILDDIR = build

# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

.PHONY: help Makefile

# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
35 changes: 35 additions & 0 deletions docs/make.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
@ECHO OFF

pushd %~dp0

REM Command file for Sphinx documentation

if "%SPHINXBUILD%" == "" (
set SPHINXBUILD=sphinx-build
)
set SOURCEDIR=source
set BUILDDIR=build

%SPHINXBUILD% >NUL 2>NUL
if errorlevel 9009 (
echo.
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
echo.installed, then set the SPHINXBUILD environment variable to point
echo.to the full path of the 'sphinx-build' executable. Alternatively you
echo.may add the Sphinx directory to PATH.
echo.
echo.If you don't have Sphinx installed, grab it from
echo.https://www.sphinx-doc.org/
exit /b 1
)

if "%1" == "" goto help

%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
goto end

:help
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%

:end
popd
Binary file added docs/source/_static/example_global_chart.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/source/_static/example_regional_chart.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading