Skip to content

extmod/zephyr_ble: Add Zephyr BLE host stack with RP2 port integration.#19

Draft
andrewleech wants to merge 9 commits intoreview/zephyr-ble-corefrom
pr/zephyr-ble-core
Draft

extmod/zephyr_ble: Add Zephyr BLE host stack with RP2 port integration.#19
andrewleech wants to merge 9 commits intoreview/zephyr-ble-corefrom
pr/zephyr-ble-core

Conversation

@andrewleech
Copy link
Owner

Summary

This adds the Zephyr BLE host as a third backend for modbluetooth, integrated as a port-agnostic extmod. The Zephyr host code is compiled against a HAL shim layer (extmod/zephyr_ble/hal/) that replaces Zephyr's kernel primitives (semaphores, work queues, timers, FIFOs, memory slabs) with MicroPython-compatible implementations. This means the Zephyr BLE host runs cooperatively on the main MicroPython task without requiring Zephyr RTOS itself.

This was motivated by limitations in the existing stacks — NimBLE doesn't have an active BLE pre-qualification and is missing some newer BLE features, while BTstack's MicroPython integration lacks pairing/bonding and L2CAP channel support. The Zephyr BLE host stack has active qualification, full feature coverage, and is under active development by multiple silicon vendors.

The RP2 port is the first integration, providing two build variants for Pico W and Pico 2 W:

  • zephyr_ble — cooperative polling from the main task (preferred)
  • zephyr_ble_freertos — HCI processing on a dedicated FreeRTOS task

Also included: a gap_unpair() API addition across all BLE backends, micropython-lib updates for aioble robustness, and bond key persistence via Python secret store callbacks.

flowchart TD
    subgraph "extmod/zephyr_ble (port-agnostic)"
        MOD[modbluetooth_zephyr.c] --> HAL[HAL shim layer]
        HAL --> WORK[work queues]
        HAL --> SEM[semaphores]
        HAL --> TIMER[timers]
        HAL --> FIFO[FIFOs]
        HAL --> H4[HCI H4 transport]
    end

    subgraph "Port integration (e.g. RP2)"
        HCI_DRV[mpzephyrport_rp2.c<br/>CYW43 HCI driver] --> H4
        POLL[main loop polling] --> WORK
    end

    subgraph "Zephyr BLE host (lib/zephyr)"
        HOST[hci_core / gatt / att / smp / l2cap]
    end

    HOST --> MOD
    HCI_DRV --> HOST
Loading

Testing

All 12 BLE multitests passing on Pico W and Pico 2 W (both zephyr_ble variant) with PYBD (NimBLE) as central:
ble_gap_advertise, ble_gap_connect, ble_characteristic, ble_gap_pair, ble_gap_pair_bond, ble_subscribe, ble_irq_calls, ble_gattc_discover_services, ble_l2cap, perf_gatt_notify, perf_l2cap, ble_gap_unpair.

Performance: ~24ms/notification (GATT), ~2184 B/s (L2CAP) on Pico W; ~25ms/notification, ~7956 B/s on Pico 2 W.

Not tested: zephyr_ble_freertos variant on Pico 2 W.

Trade-offs and Alternatives

The HAL shim layer is substantial (~3K lines) because it reimplements Zephyr kernel primitives. The alternative would be running actual Zephyr RTOS, but that would limit this to the Zephyr port only. The shim approach allows any MicroPython port with an HCI transport to use the Zephyr BLE host.

The lib/zephyr submodule adds the full Zephyr BLE host source. Only the host stack files are compiled — no kernel, no drivers, no board support. The submodule is pinned to a specific commit with two small patches (wrapper files for gatt.c and conn.c to expose static internals needed for clean teardown).

MTU is compile-time only (CONFIG_BT_L2CAP_TX_MTU=512); runtime ble.config(mtu=X) is not supported. ble_mtu.py test is skipped.

Generative AI

I used generative AI tools when creating this PR, but a human has checked the code and is responsible for the description above.

@andrewleech andrewleech force-pushed the pr/zephyr-ble-core branch 10 times, most recently from 577fa84 to eb4a81a Compare March 11, 2026 12:08
@github-actions
Copy link

github-actions bot commented Mar 11, 2026

Code size report:

Reference:  docs/esp32: Replace 'esptool.py' by 'esptool' in command line example. [e4920d6]
Comparison: extmod/zephyr_ble: Remove dead code from HAL shim files. [merge of c197acc]
  mpy-cross:    +0 +0.000% 
   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
      esp32:   +84 +0.005% ESP32_GENERIC[incl +32(data)]
     mimxrt:    +0 +0.000% TEENSY40
        rp2:  +232 +0.025% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

@andrewleech
Copy link
Owner Author

/review

@mpy-reviewer
Copy link

mpy-reviewer bot commented Mar 13, 2026

Review failed. Retry with /review.

pi-anl added 7 commits March 16, 2026 15:51
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Switch L2CAP CoC from recv+alloc_buf to the seg_recv API, which gives the
application per-PDU callbacks with manual credit control.  The old path
issued one credit per SDU, forcing the peer to wait for the full SDU to
be delivered before sending the next one.  With seg_recv, credits are
issued one per non-last PDU (allowing the peer to pipeline all K-frames
of a single SDU) and one credit per SDU from recvinto() (for the first
PDU of the next SDU), keeping at most one assembled SDU buffered.

Work around a Zephyr bug in l2cap_chan_seg_recv_rx_init() which leaves
rx.mps at zero for seg_recv channels (unlike l2cap_chan_rx_init for the
normal path), causing immediate channel disconnect on the first received
PDU.  Set rx.mps = BT_L2CAP_RX_MTU in l2cap_create_channel() and use
bt_l2cap_chan_give_credits() in accept/connect paths, matching the
pattern from Zephyr's credits_seg_recv test.

Also enable Data Length Extension (DLE) so the controller can negotiate
251-byte PDU payloads, reducing per-PDU overhead.

TX pipeline: allow up to L2CAP_SDU_BUF_COUNT-1 SDUs in flight concurrently
(tracked via tx_in_flight counter) rather than stalling after every send.

On nRF52840 dongle (PCA10059) with PYBD (NimBLE) as central:
  perf_l2cap.py before: ~2,184 B/s
  perf_l2cap.py after:  ~11,518 B/s  (5.3x improvement)
  All 11 BLE multitests pass.

Signed-off-by: Andrew Leech <andrew@alelec.net>
Replace single-SDU L2CAP accumulation buffer with a FIFO that
holds multiple SDUs.  Deep initial credit window (fills rx_buf)
allows the peer to pipeline SDUs without per-SDU credit
round-trips, which is critical for Z2Z throughput where each
credit round-trip costs 2+ connection intervals.

Add deferred L2CAP recv notification (rx_notify_pending) to
avoid re-entrancy between seg_recv_cb and Python IRQ handlers.
Each port's port_run_task must call flush_recv_notify() after
work_process completes.

Disable DLE auto-negotiation (CONFIG_BT_AUTO_DATA_LEN_UPDATE 0)
for CYW43 compatibility — CYW43 disconnects with "Instant
Passed" (0x16) when DLE is negotiated.

Add l2cap_status_cb TX kick via bt_tx_irq_raise() to unblock
queued SDUs when credits arrive.

Run codeformat.py on extmod files.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
@andrewleech andrewleech force-pushed the pr/zephyr-ble-core branch 2 times, most recently from fedbc7f to ffe840c Compare March 16, 2026 05:56
Add ZEPHYR_BLE_POLL_INTERVAL_MS define (default 128ms)
matching NimBLE convention. IRQ-driven ports use poll_now()
for immediate processing; this is a fallback for timer
housekeeping.

Change CONFIG_BT_AUTO_DATA_LEN_UPDATE to #ifndef guard so
ports with capable controllers can override via CFLAGS.

Move random data generation out of the timed window in
perf_l2cap.py and use getrandbits(8) for faster generation.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
@codecov-commenter
Copy link

codecov-commenter commented Mar 17, 2026

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.45%. Comparing base (e4920d6) to head (c197acc).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@                   Coverage Diff                   @@
##           review/zephyr-ble-core      #19   +/-   ##
=======================================================
  Coverage                   98.45%   98.45%           
=======================================================
  Files                         175      175           
  Lines                       22635    22635           
=======================================================
  Hits                        22286    22286           
  Misses                        349      349           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Cleanup unused functions, macros and debug helpers that were never called
or only conditionally compiled behind disabled feature flags. Removes dead
registry system, PSA crypto stubs, LIFO operations, and various unused
helper functions and inlines from kernel/device/config headers. Also
deletes gatt_pragma.h which is unreferenced.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants