ESP32 port

[gschorcht]

Description

This is the first working version of the RIOT-OS port for ESP32. Even though it is the first commit, most of RIOT's features are implemented and working, see the current status below. All related tests I executed from directory tests passed. Networking is realized using either

• external IEEE 802.15.4 radio modules.
• internal WiFi with AP-less ESPNOW
• internal Ethernet interface

Boards

The inital commit includes 5 board definitions:

• ESP32 DevKit and clones (generic)
• ESP-WROVER-KIT
• MH-ET Live MiniKit (pin compatible with Wemos D1 mini)
• Olimex ESP32-EVB
• Wemos Lolin D32

Current status (2018-07-25)

• context saving and restoring (implementing the core's thread inteface) is implemented and tested
• interrupt handling (the core's irq interface) is implemented and tested
• GPIO module periph_gpio is implemented and tested
• Hardware random number generator periph_hwrng implemented and tested
• CPU id module periph_cpuid implemented and tested
• timer module perihph_timer implemented and tested
• UART module periph_uart implemented and tested
• SPI module periph_spi implemented and tested
• I2C module periph_i2c implemented and tested
• PWM module periph_pwm implemented and tested
• ADC module periph_adc implemented and tested
• DAC module periph_dac implemented and tested
• RTC module periph_rtc implemented and tested
• MTD device implemented for flash and can be used with SPIFFS and VFS
• SPI RAM implemented and tested
• shell and a lot of shell-based test applications are working
• networking using external IEEE 802.15.4 transceiver modules (MRF24J40) implemented
• netdev using built-in WiFi module with the AP less ESP-NOW technology implemented (compatible to ESP8266), module esp_now
• netdev using built-in Ethernet interface (EMAC) implemented, module esp_eth
• CAN device driver implemented, module esp_can

Limitations

• The implementation uses only one core (the PRO CPU) because RIOT does not support running multiple threads simultaneously.
• WiFi works only in AP-less mode with Espressif's ESPNOW technology. Even though it is possible to connect to an AP, at the moment there are no network or transport layer functions implemented.
• Bluetooth cannot be used at the moment.
• Flash encryption is not yet supported.

Current work

• extend built-in WiFi based networking using an AP

Next steps

• implement deep sleep mechanisms
• Bluetooth based networking (don't know whether it is really possible without ESP-IDF)

Issues / PRs

depends on #10145

[gschorcht]

Just splitted the commit in several packages.

[kYc0o]

why closed?

[gschorcht]

@kYc0o Just by mistake :-( I just wanted to recommit the port to add some minor changes and the new periph_rtc module. So I resetted by branch to the master, added and committed all the files again. After forced push (at least I thought it was a push), the PR was closed and the branch is empty in origin.

[kYc0o]

wow! Ok, I hope you can restablish everything. I'm willing to review and test. However I only have a sha2017 badge which embeds a ESP-WROOM32. I hope with few modifications I can make it work with this PR.

[gschorcht]

@kYc0o I had a local version of the repository. So I could uploaded the complete commit.

Regarding you board, I can take a look what it is. The port should work with all WROVER and WROOM boards with some modifications in board definition.

[gschorcht]

@kYc0o SHA2017 Badge seems to use the touch sensors. Touch sensor inputs are not supported at the moment since RIOT has no low level interface for touch sensors. The according GPIOs can be used as ADC channels or digial I/O.

[kYc0o]

Yes, no hurries about it, the most important thing is to have the basic stuff merged, the rest can be added in follow-up PRs.

[jia200x]

@jia200x Thanks. So the CI: ready for build label could be set again?

Nein :( The workers need to be restarted. We will arrange that.

[gschorcht]

@jia200x How long will this take. Even though I have compiled all the tests with all the boards and I do not expect any surprises, I'm a little curious if murdock gives the same results.

[smlng]

I just updated all murdock workers, so it should in theory work now 🤞

[gschorcht]

@smlng compilations on haw-icc always fail with: Compiler xtensa-esp32-elf-gcc is required but not found in PATH. Does it take a while before the update of the workers become effective.

[gschorcht]

@smlng @miri64 Hm, it seems that murdock failed due to compilation errors that are not related to this PR 😟 Who should take a look?

compile/tests/cpp11_mutex/mulle:llvm
compile/tests/cpp11_mutex/iotlab-m3:llvm
compile/tests/cpp11_mutex/nrf52dk:llvm
compile/tests/cpp11_mutex/samr21-xpro:llvm
compile/tests/cpp11_mutex/nucleo-f401re:llvm
compile/tests/cpp11_mutex/slstk3402a:llvm
compile/tests/cpp11_condition_variable/nrf52dk:llvm
compile/tests/cpp11_condition_variable/nucleo-f401re:llvm
compile/tests/cpp11_condition_variable/samr21-xpro:llvm
compile/tests/cpp11_condition_variable/iotlab-m3:llvm
compile/tests/cpp11_condition_variable/mulle:llvm compile/tests/cpp11_condition_variable/slstk3402a:llvm
compile/tests/cpp11_thread/nrf52dk:llvm
compile/tests/cpp11_thread/mulle:llvm
compile/tests/cpp11_thread/nucleo-f401re:llvm
compile/tests/cpp11_thread/iotlab-m3:llvm
compile/tests/cpp11_thread/samr21-xpro:llvm
compile/examples/riot_and_cpp/iotlab-m3:llvm
compile/examples/riot_and_cpp/nucleo-f401re:llvm

[jia200x]

Who should take a look?

I'm already taking care of this.

The good news are: it seems there are no ESP32 issues 🎉

[jia200x]

@gschorcht btw thank you so much for the efforts you put in documentation

[jia200x]

Murdock is green!

[jia200x]

Ok, everything seems to be OK. Code looks good and the platform passed some extensive tests (thanks @MrKevinWeiss). Thanks for all the efforts to @gschorcht and all people involved.

Let's get this in the Release 2018.10. ACK

[miri64]

🎉

[waehlisch]

super cool!

[gschorcht]

Thank you all for your support, especially @MrKevinWeiss for the intensive testing of the port.

[emmanuelsearch]

@gschorcht and everyone else involved: congrats!

[miri64]

What is IPv6 stuff doing here oO

[gschorcht]

Good question, it is long time ago 😟

As I remember, the first packet snippet the netdev driver gets in the _send function is the IPv6 header. If the IPv6 destination address is not a link local address, the destination MAC address cannot be derived from the IPv6 destination address. Of course, I would have the same problem when privacy extensions are used. So I thought, that nib is the right way to get the mapping from an IPv6 destination address to the next hop MAC address. Maybe, I was wrong. In that case the question is how it should be.

From my understanding, IPv6 is responsible to find the next hop IPv6 address from the routing table and to use the neighbor discovery protocol to determine the next hop MAC address. This MAC address is then given as destination parameter of the MA_UNIDATA.request primitive to the L2 interface.

I couldn't figure out how this is exactly realized in GNRC.

[miri64]

This is not supposed to go in the network device. GNRC's IPv6 module should take care of all of that. If the link-layer is not multicast/broadcast capable, special care needs to be taken anyway.

[gschorcht]

This is what I feared. ESP-NOW is unfortunately a peer-to-peer network and not a meshed network 😟 That is, each message would have to be sent to each known peer as MAC unicast to simulate the broadcast. I thought I could do this hack with nib to save time by trying to figure out the peer based on IPv6 address and then only to send the message as a single unicast message. It doesn't matter if we have only 2 or 3 peers. But when the number of peers increases ...

[gschorcht]

@miri64
You said that the whole file looks like a hack. That might be, but the ESP32 port was a big project in parallel to my regular business and I was happy to get it working with WiFi at all even if it is not yet perfect. Now, where I will have some time over the comming holiday season, I would like to improve or correct it.

Therefore, I would like to ask you to give me a short guide which modules have to be used and how to get a MAC header as the first packet snippet in the _send function of the netdev driver, especially if I have Ethernet addresses, but none MAC according to IEEE 805.14 or IEEE 802.3.

I tried again to figure out how GNRC IPv6 works with netdev_eth and enc28j60, but I'm still not sure. Do I understand it correctly that the _send function of enc28j60 receives a preformatted IEEE802.3 or Ethernet II MAC header as the first packet snippet in the parameter iolist? If so, would it be an appropriate way to change the esp_now netdev driver to be rather an adoption layer that emulates an Ethernet MAC layer over ESP-NOW and simply to use it like an netdev_eth device?

[TimoRoth]

You are right about that. The raw netif explicitly strips that header away right here: https://github.com/RIOT-OS/RIOT/blob/master/sys/net/gnrc/netif/gnrc_netif_raw.c#L97

[TimoRoth]

I got an initial (but yet fully untested beyond compiling it) version of the rework of this on https://github.com/TimoRoth/RIOT

[miri64]

Am I right that the reason why ESP-NOW gets the IPv6 header as the first header instead of the netif header with the required L2 information because it creates a netif thread of type raw with function gnrc_netif_raw_create?

Yepp. (also as a side-note: the current gnrc_netif concept is a bit different and younger than the state of my master thesis)

[miri64]

But to restate my comment from #10531 (comment):

The gnrc_netif_raw implementation was intended for link-layers without any addresses (e.g. SLIP) or were access to the link-layer address is not necessary.

[miri64]

I got an initial (but yet fully untested beyond compiling it) version of the rework of this on https://github.com/TimoRoth/RIOT

Now provided as PR to discuss on #10581.

[miri64]

Just as a hint for future reference: the esp-now and esp-wifi part could have been stripped out of this PR and put into a separate one. Discussions like the one above could have been avoided otherwise.

[gschorcht]

@miri64 Thanks 😄