Terkin Datalogger backlog

Datalogger 2020

S1

  • Optional Zeitstempel in MQTT/JSON-Nachricht einfügen.

  • Verschiedene Sendezyklen pro Telemetrie-Eumel ermöglichen.

  • “LoRaDriverDragino.ensure_connectivity()” non-blocking gestalten.

  • Sensor-Wrapper für PyTrack, Gpsd und 2xSolarlader erschließen.

S2

  • Cayenne-Mapping: Use dataframe.readings.

  • Sensor-Gruppen per Tags definieren.

  • Sensor-Metadaten einführen. Beispiel: Timestamp pro Sensor-Reading.

  • data_in => data_out

S3

  • /backup-Verzeichnis optional ansprechen

  • Exceptions auf CPython abfangen

  • Installation von mpy-cross auf RaspberryPi aussparen

  • Sensors: Observation einführen, SensorReading => Reading umbenennen

  • Telemetry: Sequans Modem erschließen

  • Telemetry: settings.telemetry_name honorieren

  • Improve formatting of “Sensor data” output

  • Setup: Tracer-MT5 code auslagern

  • Setup: gpsd code auslagern

  • Setup: Debian-Paket bauen?

  • Setup: Abhängigkeiten klären: python-spidev python3-spidev + pip spidev!?

  • Setup: /boot/config.txt overlay und /etc/default/gpsd-Konfiguration automatisieren

  • Setup: dpkg-reconfigure UI?

  • Doku bei https://community.hiveeyes.org/t/uberarbeitung-des-terkin-readme-im-repository/3152/5 bzw. https://github.com/hiveeyes/terkin-datalogger/blob/main/doc/cpython.rst aktualisieren und veröffentlichen

  • Update credits

S4

S5

S6

S7

  • Make “settings-user.json” work

  • Update Annapurna firmware images

  • Send annotations

  • Improve tests using

  • Detect and mitigate connection-loss with LTE-connectivity

  • Add reading /sys/class/thermal/thermal_zone0/temp

  • Tweak MQTT configuration to publish just to designated MQTT topic

  • neocon in “make console” einbauen

  • Set system time from NTP or GPS

Documentation

Datalogger 2019

Prio 0.7

Prio 0.8

Prio 0.9

[main.py] INFO: Starting Terkin Datalogger
   18.3435 [terkin.configuration     ] INFO   : Starting TerkinConfiguration on path "/flash"
   18.3644 [terkin.configuration     ] INFO   : Ensuring existence of backup directory at "/flash/backup"
Unhandled exception in thread started by <bound_method>
Traceback (most recent call last):
  File "network/ip.py", line 24, in start_real
OSError: Network card not available

Prio 1

Prio 1.1

  • [o] Publish system events to MQTT

  • [o] Subscribe to MQTT downlink channel

  • [o] Unlock NVRAM storage as ConfigurationSettings overlay

  • [o] Add named fields based on NVRAM overlay

  • [o] OneWire sensor enumeration - display lexographically sorted?

  • [o] DEBUG mode

Prio 1.2

Prio 1.3

Prio 1.4

Prio 1.5

  • [o] Store-and-forward when no connectivity

  • [o] Add AP mode and webserver

  • [o] Better BME/BMP libraries

  • [o] Bli, bla, blubb

  • [o] Propagate last error message from telemetry subsystem

    into intermediary status outcome and display to user.

  • [o] Save from radio.py:

    # Todo: What about when coming back from sleep?
# Needed to avoid losing connection after a soft reboot
# if True or machine.reset_cause() != machine.SOFT_RESET:
import pycom

Prio 2

  • [o] Don’t upload the “terkin” library each time.

  • [o] Improve tooling by adding file watcher or even delta patching

  • [o] Improve display of “Networking address” by decoding mac address values

  • [o] Add configuration variant based on JSON file

  • [o] Sensor data model: Introduce Measurement (single) and Reading (bunch) objects

  • [o] Status and sensor announcement

  • [o] Reconnect to WiFi and MQTT when dropping off, use exp. backoff?

  • [o] Periodic servicing tasks for NetworkManager

  • [o] Add MQTT-based runtime configuration like mqtt://daq.example.org/.../settings.json or .../rpc/request vs. .../rpc/response

  • [o] Is utime actually the same as time?

  • [o] Publish sensor configuration at first time telemetry contact

  • [o] Implement access to Switches: https://micropython.readthedocs.io/en/latest/pyboard/tutorial/switch.html

  • [o] Use Timers: https://micropython.readthedocs.io/en/latest/pyboard/tutorial/timer.html

  • [o] Put dotty_dict into dist-packages

  • [o] Don’t submit data when null:

    workbench/testdrive/area-38/fipy-amo-02-mqtt-lpp/data.lpp (null)

Prio 3

Prio 4

  • [o] Unlock frozen modules: Upload .mpy code through mpy-cross

  • [o] Upload watcher

  • [o] Add DS18B20: https://github.com/pycom/pycom-libraries/tree/master/examples/DS18X20

  • [o] WiFi soft reset re. if machine.reset_cause() != machine.SOFT_RESET:

  • [o] Improve the AP mode:

    [0.06439157] Starting networking
WiFi STA: Starting connection
WiFi STA: Connect failed: list index out of range. Switching to AP mode.
2 fipy-wlan-42bc (3, 'www.pycom.io') 0
Networking established
[3.663849] Starting telemetry

  • [o] Publish retained status message to MQTT like beradio-python:

    hiveeyes/fe344422-05bf-40f2-a299-fbf4df5d7e2b/vay55/gateway/status.json {"status": "online", "program": "beradio 0.12.3", "date": "2019-03-07T19:38:28.462900"}

  • [o] Reenable WiFi AP mode

  • [o] How to use uPy module “urequests”?:

    # Problem: "urequests" does not work with SSL, e.g. https://httpbin.org/ip
# micropython -m upip install micropython-urequests
#import urequests
  • [o] Check out “Firmware over the air update”:

    https://github.com/pycom/pycom-libraries/blob/master/examples/OTA/OTA_server.py

  • [o] Add network name to “Already connected”

  • [o] Automate cayennelpp installation https://github.com/smlng/pycayennelpp

  • [o] Assistant for configuring serial_port in config.mk. Optionally use environment variable!?

  • [o] Use more information from WiFi station:

    'antenna', 'ap_sta_list', 'auth', 'bandwidth', 'bssid', 'callback', 'channel', 'connect', 'country', 'ctrl_pkt_filter', 'deinit', 'disconnect', 'events', 'hostname', 'ifconfig', 'init', 'isconnected', 'joined_ap_info', 'mac', 'max_tx_power', 'mode', 'promiscuous', 'scan', 'send_raw', 'ssid', 'wifi_packet', 'wifi_protocol']

Done

  • [x] Solid configuration system

  • [/] Unlock NVRAM storage:

    > Set the value of the specified key in the NVRAM memory area of the external flash.
> Data stored here is preserved across resets and power cycles.
> Value can only take 32-bit integers at the moment.

  • [/] class NvsStore: https://forum.pycom.io/topic/2775/keeping-state-when-awaking-from-sleep

  • [x] Use pure-Python “urllib” without dependency to “libpcre”

  • [x] Handle “Connection to MQTT broker failed or lost”

  • [x] Multi-network WiFi

  • [x] Fix console crasher when running on Windows

  • [x] Release version 0.1.0

  • [x] Real sensors already

  • [x] Add release tooling

  • [x] Add snapshot of dist-packages folder as asset to each release on GitHub

  • [x] Add appropriate logging

  • [x] Improve logging by adding stacktrace printer

  • [x] Add some examples

  • [x] Report about which telemetry targets did actually work when submitting data (True / False)

  • [x] Improve logging: Suppress logging of sensible information like password, application_key, application_eui, mac or ifconfig

  • [x] Switch to LittleFS

  • [x] Add deep sleep

  • [x] Log wakeup type

  • [x] Improve formatting of mac addresses

  • [x] Power saving

    • [x] Turn off logging

    • [x] Turn off heartbeat of RGB-LED

    • [x] Speed-up WiFi connection by not scanning at all.

      In order to achieve that, scan once and remember auth-mode in NVRAM.

    • [x] Fix HX711 power down re. spec

    • [x] Activate internal pull-up for HX711 PD_SCK in deep sleep mode with “pin hold”. https://docs.pycom.io/firmwareapi/pycom/machine/pin.html#pinholdhold

    • [x] Turn off serial interface completely

    • [x] Tame LED-RGB

  • [x] Activate Watchdog Timer

  • [x] make recycle-ng needs network!?

  • [x] WiFi.is_connected would also return True when AP is up!!!

  • [x] Make “make help” point to “Operate the …”

  • [x] Implement real “light sleep” “in light sleep mode the current consumption on a Lopy is 3.5 mA with RTC peripherals ON” https://forum.pycom.io/topic/3351/new-development-firmware-release-v1-19-0-b1/3

  • [x] Add configuration for external WiFi antenna

  • [x] Properly calculate sleep time from interval and duty cycle duration

  • [x] Connect to Beep

Sandbox environment

Upload and reset

  • [o] How to run program with soft reset instead of hard reset?

  • [o] How to run multiple repl commands at once?

  • [o] Improve build time

    • make recycle scopes=full

    • make recycle scopes=main,hiveeyes

    • make recycle scopes=main,terkin

  • [o] Improve tests

    • Run against non-standard MQTT port (1888)

    • UDP API

    • HTTP API

    • WiFi-HTTP telemetry

    • Pure-Python DS18B20 driver

    • State machine

Documentation

Other projects

Misc

workbench/testdrive/area-38/fipy-amo-02-mqtt-json/data.json {"temperature_0": 42.42, "temperature_1": -84.84}
workbench/testdrive/area-38/fipy-amo-02-mqtt-json/data.lpp AGcBqAFn/LA=

User interface

Firmware update output

Erased 2 MiB in 15.28 seconds
Erased 4MB device flash fs in 1.22 second
Wrote 20.95 KiB from bootloader.bin in 1.11 second
Wrote 3 KiB from partitions.bin in 0.08 seconds
Wrote 1.66 MiB from fipy.bin in 54.4 seconds
Wrote 4 KiB from config in 0.1 seconds
Device ID: 807D3AC2DE44
LoRa MAC: 70B3D54992DBE31D
Sigfox ID: 004D4881
Sigfox PAC: 211AC57838BF7C29