Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 85 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
name: CI

on:
push:
pull_request:
workflow_dispatch:

jobs:
build-and-test:
name: Build and test
runs-on: ubuntu-latest
permissions:
contents: read

Comment thread
coderabbitai[bot] marked this conversation as resolved.
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
persist-credentials: false

- name: Build image
run: docker build --tag micropython-linux:test .

- name: Substrate conformance (esp32 fakes vs contract)
run: |
docker run --rm \
-v ${{ github.workspace }}:/work \
-w /work \
micropython-linux:test /work/tests/substrate-conformance.py

- name: Checkout micropython-wifimanager
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
repository: mitchins/micropython-wifimanager
# Pinned to a specific commit (not a branch) for deterministic CI.
# Must be a revision that includes the API the suite exercises
# (_scan_available_networks/_build_connection_candidates/_connect_candidates/
# _load_config) — these post-date the v1.0.2 release tag.
ref: 835861c0ed10b170bcf29a171e1eb11177131084
path: micropython-wifimanager
persist-credentials: false

- name: WifiManager regression on substrate (MicroPython runtime)
run: |
docker run --rm \
-v ${{ github.workspace }}:/work \
-w /work \
-e WIFIMANAGER_SRC=/work/micropython-wifimanager \
micropython-linux:test /work/tests/wifimanager-status-regression.py

- name: Smoke test image
run: docker run --rm micropython-linux:test -c "import sys; print(sys.version)"

publish:
name: Publish image
if: github.event_name == 'push' && github.ref == format('refs/heads/{0}', github.event.repository.default_branch)
needs: build-and-test
runs-on: ubuntu-latest
permissions:
contents: read
packages: write

steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
persist-credentials: false

- name: Build image
run: docker build --tag micropython-linux:test .

- name: Log in to GHCR
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Publish image to GHCR
run: |
IMAGE=$(echo "ghcr.io/${{ github.repository }}" | tr '[:upper:]' '[:lower:]')
docker tag micropython-linux:test "$IMAGE:latest"
docker tag micropython-linux:test "$IMAGE:${{ github.sha }}"
docker push "$IMAGE:latest"
docker push "$IMAGE:${{ github.sha }}"
15 changes: 0 additions & 15 deletions .travis.yml

This file was deleted.

43 changes: 26 additions & 17 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,20 +1,29 @@
FROM debian:stretch-slim
FROM debian:bookworm-slim

ARG MICROPYTHON_VERSION=v1.28.0

RUN apt-get update && \
apt-get install -y build-essential libffi-dev git pkg-config python3 && \
rm -rf /var/lib/apt/lists/* && \
git clone https://github.com/micropython/micropython.git && \
cd micropython && \
cd mpy-cross && \
make && \
cd .. && \
cd ports/unix && \
make submodules && \
make && \
make test && \
make install && \
apt-get purge --auto-remove -y build-essential libffi-dev git pkg-config python3 && \
cd ../../.. && \
rm -rf micropython
apt-get install -y --no-install-recommends \
build-essential \
ca-certificates \
git \
libffi8 \
libffi-dev \
pkg-config \
python3 \
Comment on lines +6 to +13

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The unix port of MicroPython dynamically links to libffi for its ffi module. When libffi-dev and python3 are purged with --auto-remove (lines 21-26), the runtime library libffi8 (which is installed as a dependency of libffi-dev) will be automatically removed if no other package depends on it. This will cause the micropython binary to fail at runtime with a shared library loading error (e.g., libffi.so.8: cannot open shared object file).

To prevent libffi8 from being auto-removed, explicitly install libffi8 in the apt-get install list.

    apt-get install -y --no-install-recommends \
      build-essential \
      ca-certificates \
      git \
      libffi-dev \
      libffi8 \
      pkg-config \
      python3 \

&& git clone --depth 1 --branch ${MICROPYTHON_VERSION} https://github.com/micropython/micropython.git \
&& cd micropython \
&& cd ports/unix \
&& make submodules \
&& make \
&& make install \
&& cd / \
&& rm -rf micropython \
&& apt-get purge --auto-remove -y \
build-essential \
git \
pkg-config \
python3 \
&& rm -rf /var/lib/apt/lists/*
Comment thread
coderabbitai[bot] marked this conversation as resolved.

CMD ["/usr/local/bin/micropython"]
ENTRYPOINT ["/usr/local/bin/micropython"]
101 changes: 81 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,93 @@
# docker-micropython-linux
Docker image that comes loaded with the unix (linux in this case) port of micropython.

#### Why use this? Simple:
A pinned **MicroPython runtime in Docker** plus a small **ESP32 test substrate**, for
running deterministic CI tests of MicroPython libraries — without a board, and without
the false confidence of testing under CPython.

* Running automated tests against libraries intended for micropython (you *can* use python3, but there are edge cases).
* Experimenting with micropython without any complexity of building yourself or overhead of buying a board first.
* There wasn't an existing and well-maintained image.
![CI](https://github.com/mitchins/docker-micropython-linux/actions/workflows/ci.yml/badge.svg)

The image is based off the official Debian-slim (stretch) because it's fairly slim (not as slim as alpine, but that can be a headache to build).
## Why this exists

#### Getting Started
MicroPython is **not** CPython. The standard library is a subset and behaves
differently: there is no `os.path`, no `types`, `base64` may be absent, `os.environ`
is `os.getenv`, and module surfaces differ. A library "tested" under `python3` can pass
its suite and still break on device. This repo runs your library's logic under the
**actual MicroPython interpreter** (the unix port), pinned to a known version, in a
throwaway container.

Providing you have access to Docker, you can run the latest version quite easily by:
On top of that it ships a **scriptable fake of the ESP32 SDK** (`network`, `machine`)
so you can exercise Wi-Fi/state/pin logic deterministically — the fake provides the
*shape* (signatures, return types, exceptions, constants, identity captured from the
real firmware); your test provides the *behaviour*.

docker run -it mitchins/micropython-linux
MicroPython v1.9.4-403-g81e320ae on 2018-07-21; linux version
Use Ctrl-D to exit, Ctrl-E for paste mode
>>> ^C
**Scope, honestly:** this targets **logic/framework correctness** (connection policy,
status handling, config parsing, callbacks, pin I/O sequencing) — not RF, timing, or
electrical behaviour. It is a high-signal, low-flake complement to on-device testing,
not a replacement.

Tags are available on the Docker Hub listing
## The image

If you check the Dockerfile, you will see it starts up micropython automatically which is stored in /usr/local/bin/micropython
Pulled from GitHub Container Registry (built and published by CI — see below):

To install something else, you can:
docker run -it ghcr.io/mitchins/docker-micropython-linux:latest
MicroPython v1.28.0 on ...; linux version
>>>

# micropython -m upip install micropython-unittest
Installing to: /root/.micropython/lib/
Warning: pypi.org SSL certificate is not validated
Installing micropython-unittest 0.3.2 from https://files.pythonhosted.org/packages/9a/42/41057b8da94414a17f7028ee08035c3d945befebddc76d58988067ddaf0f/micropython-unittest-0.3.2.tar.gz
It builds the unix port of MicroPython from source on `debian:bookworm-slim`. Override
the version at build time:

That's about it, the goal is to maintain the latest and stable tagged versions so regressions can be tested as needed, or experimented features accessed.
docker build --build-arg MICROPYTHON_VERSION=v1.28.0 -t micropython-linux .

The container's entrypoint is `micropython`, so arguments pass straight through:

docker run --rm ghcr.io/mitchins/docker-micropython-linux:latest -c "import sys; print(sys.version)"
docker run --rm -v "$PWD":/work -w /work ghcr.io/mitchins/docker-micropython-linux:latest /work/script.py

## The ESP32 substrate

In [`substrate/`](substrate/):

| File | Purpose |
|---|---|
| `CONTRACT-esp32-v1.28.md` | The SDK contract, captured from the esp32 port C source (MicroPython v1.28.0) + ESP-IDF v5.5.1 |
| `network.py` | `WLAN` fake — singleton-per-interface, `bytes` SSIDs, real `STAT_*` values, scriptable + spy |
| `machine.py` | `Pin` fake — `int` reads, readable `OUT`, scriptable inputs + spy |
| `aiotest.py` | `run_iterations()` — drives an async loop (e.g. `manage()`) deterministically, no wall-clock waits |

The fakes are *scriptable*, not emulated. A test programs outcomes and spies on calls;
it never authors a mock:

```python
import sys; sys.path.insert(0, "substrate")
import network

sta = network.WLAN(network.STA_IF)
sta.active(True)
sta.program_scan([{"ssid": "Home", "bssid": b"\x0a\x0b\x0c\x0d\x0e\x0f", "rssi": -50}])

# ... exercise your library against `network` ...

assert sta.call_count("connect") == 1
assert sta.last_call("connect").kwargs["bssid"] == b"\x0a\x0b\x0c\x0d\x0e\x0f"
```

See [`tests/`](tests/) for a full worked example: `wifimanager-status-regression.py`
tests [micropython-wifimanager](https://github.com/mitchins/micropython-wifimanager)
against the substrate (including driving its async `manage()` loop), and
`substrate-conformance.py` pins the fakes to the captured contract so they can't drift.

## CI

[`.github/workflows/ci.yml`](.github/workflows/ci.yml) on every push/PR:

1. Build the image
2. **Substrate conformance** — fakes vs `CONTRACT-esp32-v1.28.md`
3. **Library regression** — wifimanager suite in the MicroPython runtime (against a pinned library revision)
4. Smoke test
5. On push: **publish** the image to `ghcr.io/mitchins/docker-micropython-linux`

## Status / roadmap

- Pinned to MicroPython **v1.28.0**, ESP32 profile.
- Deferred: ESP8266 profile; broader `machine` (`irq`, ADC, I2C/SPI) — added when a real
consumer needs them. The substrate establishes the pattern; it is single-target today.
101 changes: 101 additions & 0 deletions substrate/CONTRACT-esp32-v1.28.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# ESP32 SDK fake — conformance contract

**Pinned to:** MicroPython **v1.28.0**, **esp32** port.
**Source of truth:** captured from the esp32 port C bindings at the `v1.28.0` tag
(`ports/esp32/network_wlan.c`, `ports/esp32/modnetwork.h`,
`ports/esp32/modnetwork_globals.h`, `ports/esp32/machine_pin.c`) — **not** docs,
not the unix port (which exposes neither `network.WLAN` nor esp `machine.Pin`).

The fakes in this directory must "walk and quack" like these surfaces: the library
under test must not be able to tell the difference at the call boundary. The fake
supplies *shape* (signatures, return types, exceptions, constants, identity); the
**test supplies behaviour** via the control surface.

---

## `network` module constants

| Symbol | Value | Provenance / stability |
|---|---|---|
| `STA_IF` | `0` | micropython-owned, stable |
| `AP_IF` | `1` | micropython-owned, stable |
| `STAT_IDLE` | `1000` | `modnetwork.h` enum — **stable** |
| `STAT_CONNECTING` | `1001` | **stable** |
| `STAT_GOT_IP` | `1010` | **stable** |
| `STAT_BEACON_TIMEOUT` | `200` | `WIFI_REASON_BEACON_TIMEOUT` (ESP-IDF v5.5.1) |
| `STAT_NO_AP_FOUND` | `201` | `WIFI_REASON_NO_AP_FOUND` (ESP-IDF v5.5.1) |
| `STAT_WRONG_PASSWORD` | `202` | `WIFI_REASON_AUTH_FAIL` (ESP-IDF v5.5.1) |
| `STAT_ASSOC_FAIL` | `203` | `WIFI_REASON_ASSOC_FAIL` (ESP-IDF v5.5.1) |
| `STAT_CONNECT_FAIL` | `203` | **aliases `STAT_ASSOC_FAIL`** on esp32 (same int, per micropython source) |
| `STAT_HANDSHAKE_TIMEOUT` | `204` | `WIFI_REASON_HANDSHAKE_TIMEOUT` (ESP-IDF v5.5.1) |

Failure-code source: `wifi_err_reason_t` in
`components/esp_wifi/include/esp_wifi_types_generic.h` at the **esp-idf `v5.5.1`** tag —
the IDF version MicroPython v1.28.0 recommends (`ports/esp32/README.md`). For reference,
`WIFI_REASON_CONNECTION_FAIL = 205` (which MicroPython's `status()` also reports as
wrong-password → `202`).

**Two-zone confidence.** The happy-path codes (`1000/1001/1010`) are MicroPython's
own and stable across versions. The failure codes above are ESP-IDF enum values pinned
to v5.5.1; they are version-sensitive, so re-capture from the matching IDF header if the
image's MicroPython/IDF pairing changes. On the rp2/cyw43 port these constants have
entirely different values (`CYW43_LINK_*`) — which is why this fake is esp-only and
version-pinned.

## `network.WLAN`

- `WLAN(interface=STA_IF)` returns the **same singleton per interface**
(esp32 `make_new` returns static `wlan_sta_obj` / `wlan_ap_obj`). Bad id → `ValueError`.

| Method | Signature | Returns | Raises |
|---|---|---|---|
| `active([bool])` | optional arg | `bool` | `OSError` on driver error |
| `connect(ssid=None, key=None, *, bssid=None)` | `ssid`/`key` positional; `bssid` **kw-only**, 6 bytes | `None` | `ValueError` (bad bssid len), `OSError` |
| `disconnect()` | — | `None` | `OSError` |
| `status([param])` | no-arg, or `'rssi'` / `'stations'` | no-arg → **int** (STA only; AP → `None`); `'rssi'` → int; `'stations'` → list of 1-tuples of `bytes` | `ValueError` (unknown param), `OSError` |
| `scan()` | — | **list of 6-tuples** | `OSError("STA must be active")` if STA inactive |
| `isconnected()` | — | `bool` | — |
| `ifconfig([tuple])` | get/set | 4-tuple `(ip, mask, gw, dns)` of `str` | — |
| `config(key)` / `config(**kw)` | get/set | get → value; set → `None` | — |

**`scan()` tuple shape (load-bearing):**
`(ssid: bytes, bssid: bytes, channel: int, RSSI: int, security: int, hidden)`
- `ssid` and `bssid` are **`bytes`**, never `str`.
- on esp32 `hidden` is **always `False`** (never populated; `// XXX hidden?` in source).

## `machine.Pin`

- `Pin(id, mode=None, pull=-1, *, value=None, drive=None, hold=None)`.
Bad id → `ValueError("invalid pin")`.

| Member | Behaviour |
|---|---|
| `value([x])` | no-arg → **int `0`/`1`** (`gpio_get_level`); with arg → `None` (sets) |
| `pin(x)` | call form ≡ `value(x)` |
| `on()` / `off()` / `toggle()` | return `None` |
| `init(...)` / `irq(...)` | exist (`irq` deferred for day-zero) |

**Mode constants:** `IN`=`GPIO_MODE_INPUT` (`1`), **`OUT`=`GPIO_MODE_INPUT_OUTPUT` (`3`)**,
`OPEN_DRAIN`=`..._OD` (`7`). Because `OUT` is *input-output* on esp32, **reading an
output pin returns the last written level** — the fake replicates this. Pulls:
`PULL_UP`, `PULL_DOWN`.

---

## Confirmed quack-leaks the fake must honour

1. `scan()`/`status('stations')` return **`bytes`**, not `str`.
2. `WLAN(iface)` is a **singleton per interface** (identity is observable).
3. `scan()` `hidden` element is **always `False`** on esp32.
4. `Pin.value()` returns **`int`**, not `bool`.
5. `Pin.OUT` is **readable** (read-back of written level).
6. `STAT_*` failure codes are **esp/IDF-specific** and `CONNECT_FAIL == ASSOC_FAIL`.

## Provenance summary

- `network`/`machine` shapes, constants, return types, exceptions: MicroPython esp32
port at the **v1.28.0** tag.
- Failure `STAT_*` integers: ESP-IDF **v5.5.1** `wifi_err_reason_t`.

Re-capture both if the image bumps its MicroPython version (and with it the
recommended IDF version).
Loading
Loading