Seeking Advice on Integrating LoRa Project into ESP Easy

[warwickchapman]

Hello Team ESPEasy,

I'm considering utilising the ESPEasy platform to incorporate an existing vanilla .cpp code base of LoRa Tx/Rx capabilities. Before diving into the development process, I wanted to reach out to this community for some advice.

My project involves using the ESP8266 hardware with the sandeepmistry/LoRa library to achieve LoRa-based transmit and receive. The goal is to create a system that can reliably send and receive data over LoRa, and I'm particularly interested in using ESPEasy for WiFi management and creating a simple web dashboard and control panel.

Here are a few specific points where your experience and insights would be invaluable:

Feasibility and Compatibility: Have any of you integrated LoRa functionality, particularly using the sandeepmistry/LoRa library, with ESP Easy? If so, how compatible is it with the ESP Easy architecture?

Custom Plugin Development: What challenges might I face in developing a custom plugin for LoRa within ESP Easy? Are there any particular aspects of ESP Easy’s plugin system that I should be aware of?

Web Interface Customisation: How feasible is it to modify ESP Easy’s web interface to include custom UI elements for LoRa control and data visualisation? I would want to manage WiFi, manage a few configuration options and provide a simple status dashboard and a toggle or two.

Performance Considerations: Are there any known performance implications or limitations when integrating additional libraries like LoRa into ESP Easy, especially concerning the ESP8266’s resource constraints?

Best Practices and Tips: For those who have undertaken similar integrations, are there best practices, tips, or pitfalls that you can share to help guide this project towards success?

Your collective advise is appreciated.

Best wishes from sunny South Africa

Warwick

[Ath]

Hey Warwick,

Can only answer a few of your questions, but the most import one probably is: Don't try to do this on an ESP8266, but get an ESP32 or ESP32-S3 with 16 MB Flash. That's the future of ESP development in general, ESP8266 is too limited in resources to build new, and rather demanding like this, projects on.

We have a development guide in the documentation (scroll up for some basic setup using VSCode, leave out any request for Arduino IDE, that's not very usable for ESPEasy development any more, unless you really know what you are doing). This development guide is mostly aimed at Plugin development, but your LoRa project should probably be a Controller.

There is already the LoRa TTN - RN2483/RN2903 controller (that I never used myself nor developed on...), maybe that can be an inspiration, or possibly it could be extended to support your request?

I expect TD-er to chime in, as he has more experience with LoRa stuff

[warwickchapman]

Ah damn. I have already made quite a number of boards using the ESP8266.

I'll take a look at changing them to ESP32 going forward and I'll build a raw dashboard and manually implement WiFi for now.

But, I am interested in the rest of the feedback.

Thank you for your advice.

[TD-er]

In ESPEasy the exchange of data is not done via a "Plugin" but rather a "CPlugin" (as called in the source code files) or as we call it in the user interface "Controller".

I have not used the library you mentioned.
I only have implemented support for RN2483 as that module already has a fully certified LoRaWAN stack on board.

Typically those cheap "LoRa" modules out there only have a SX12xx chip for the LoRa radio transceiver.
But that's not yet LoRaWAN.
For LoRaWAN you need an additional LoRaWAN stack.
It is really hard to pick a proper software LoRaWAN stack which will be compliant with the LoRaWAN specifications.
For example when looking at "the" LMIC stack... There are so many variations, forks etc. of LMIC, you can't really speak about "the" LMIC stack.

So that's the reason I started with those RN2483 modules as it already is quite a steep learning curve to get started with LoRaWAN, thus eliminating extra unknowns sounded like a good start to me.

About the ESP8266 vs ESP32 part...
You can use ESP8266, but I would certainly advice against it for new designs as the amount of RAM and the rather limited max. sketch size are really a limiting factor which will bite you over and over again.
I think your own time is more valuable than a few dozen ESP8266 modules.


I strongly advice you to look into LoRaWAN a bit more before starting such an ambituous project.
It is for sure a really nice technique with lots of possibilities not seen in many other communication protocols. (only NB-IoT can be considered an alternative, as long as you're willing to pay a fee to a provider)

One thing about LoRaWAN that people often don't realize is that you can't send a packet from a gateway to a node.
You can only queue downlink messages which will be sent by the gateway as soon as an uplink packet is received by this gateway.
So if you only send out sample data every 15 minutes as a LoRaWAN uplink message then you might only send something to such a node as downlink within the RX-window after this uplink has been received.

Also you can only transmit for 1% of the time.
The "Spreading Factor" can be seen as some kind of indicator on how much time it may take to send out your data.
Let's assume sending some data packet at SF7 takes 100 msec to send.
At SF8 this will take 200 msec, at SF9 it takes 400 msec, etc.
Every step up, the time needed doubles.
And in this time you really can't send much data.
Typically your message should be less than 50-ish bytes.

This is just a very very short primer on all there is to know about LoRaWAN, so maybe you understand why I call it a "steep learning curve"
This has all been managed in the LoRaWAN RN2483 controller.
But this one only supports sending data, not yet receiving downlink messages.

[warwickchapman]

Thank you for the reply.

I am using the SX1278 on a Ra02 along with an ESP12F that has 4MB of memory.

I am not using LoRaWAN but pure LoRa. The application doesn’t require LoRaWAN; it is rather just a local controller / remote switch(es) scenario where the Rx devices are occasionally receiving a command to switch.

That said, I will take a look at the chipset you specified.

PS. I chose LoRa because I wanted the range. Eg. Controller Tx is 400m away from the switch Rx on a farm.

The current hardware solution works very well and the users are happy with a plug and play solution. But I’ve been trying to avoid writing a UI from scratch and ESP-DASH and ESP-UI are not suitable.

Warwick

[TD-er]

OK, if you plan on using LoRa instead of LoRaWAN, then I guess you could have a look at how the C013 (ESPEasy p2p controller) is being used.
I guess that would be the closest to what you plan on using it for.

Just keep in mind that sending data via LoRa/LoRaWAN is really slow, so you must find some way to make this a background process or else everything will be halted and you will see lots of other issues.

I don't know whether this SX1278 does have some buffers or at what level you need to handle the data.

Also keep in mind the SX1278 is for the 433 MHz band.
This band is not really a good choice as there are lots and lots of devices spamming this part of the spectrum.
So keep in mind to implement some proper timeout and retry procedure and don't expect immediate response as it is a rather slow protocol and thus long transmission times and as a result higher probability to get interrupted by others like weather stations etc.

[chemmex]

I only have implemented support for RN2483 as that module already has a fully certified LoRaWAN stack on board.

Does it use AT commands or some kind of proprietary serial protocol?

[TD-er]

It is hard to tell if it is a proprietary protocol as it is almost literally the implementation of the LoRaWAN standard.
Anyway it is extremely well documented in the documentation provided for the modules.

But still adding support out of the blue for LoRaWAN is quite a steep learning curve, so that's why I put it all in the ESPEasy controller.

[chemmex]

If I understood correctly from the docs, there is no formal protocol but rather a set of ASCII commands for module configuration, which is making it proprietary as opposed to more or less standardized AT command set.

Why I am raising this question is that I think it would be great to have a controller template for various AT-based modules, such as GSM/NB-IoT, which there are plenty of around. Not necessarily a whole library like Blynk, but an interface to add different AT-controlled devices

[TD-er]

I get why you think this would be a good idea.
However LoRaWAN and GSM are extremely different in how they operate, thus making an universal interface for it would either be as basic as "SerialPort" or completely unmanageable as there is hardly any overlap in code.

I don't know enough about NB-IoT (yet) but I guess those might be somewhat similar to warrant a generic interface.

What would be great to have however is a generic interface for the bare modules only using SX12xx chips and complete modules with a LoRaWAN stack like the RN2483.
Thus any stack implementation out there offering the same interface as the RN2483 does with maybe some very specific functions to get the state and restore the state.

Some things that you really should store are the used keys, the nounce and packet counters. (and maybe some more)
If you loose those, you might need to perform a new OTAA, which requires you to be relatively close to a gateway so the gateway can receive the join request and reply to it (and the node to receive the join data).

For example there is some node out there close to my gateway which tries to join every minute.
But it tries to join to a KPN-address range so the TTN gateway cannot fulfill this join request.
It keeps on trying to join, but I guess it will never succeed.

The other way is to use ABP, which is somewhat similar to DHCP vs. static IP. (OTAA being like DHCP and ABP being like static IP)
But you also need to keep track of the packet count or else new packets will be dropped if you reuse the same count as before.
And if you just store a count ever 1000 packets and then add 1000 when you lost the count, just to be sure, you may cycle through the max. packet count quite fast.

[warwickchapman]

OK, if you plan on using LoRa instead of LoRaWAN, then I guess you could have a look at how the C013 (ESPEasy p2p controller) is being used.

Given a cursory review of _C013.cpp, and having worked enough with sandeepmistry/arduino-LoRa to have a production codebase in place, I think that the task of altering C013 for LoRa might look like the following:

1. Include necessary LoRa library headers and remove UDP-related headers.

2. Update/replace initialisation code to set up the LoRa module instead of the UDP setup.

3. Rework C013_sendUDP, C013_Receive, C013_SendUDPTaskInfo, and C013_SendUDPTaskData functions to use LoRa instead of UDP.

4. Replace the logic in functions that send data (C013_SendUDPTaskInfo and C013_SendUDPTaskData) to use LoRa's sending methods. Ensure data formatting and packet structure are compatible with LoRa communication.

5. Update the receiving function C013_Receive function to handle data received through LoRa. Implement logic to parse and process the incoming data correctly.

6. Incorporate error handling specific to LoRa communication. Update status feedback mechanisms to reflect the state of the LoRa module.

7. Create / alter the documentation to reflect the changes made.

Thoughts?

[Ath]

Thoughts?

Please copy that source to _C021.ino, rename all C013 references to C021 and match the controller name accordingly, so it can be used as a separate Controller.
We keep a list of planned Plugins and Controllers on github, in this issue.
I'll add this to the list

[TD-er]

Yep, as Ton already suggested, please do so with new controller ID.

[warwickchapman]

Ok, I will start this in January.

[michcfr]

Hello,
Can ESPEasy coupled with the RN2483 module run also as (single Channel) LORAWAN gateway?
Regards,
Michel

In ESPEasy the exchange of data is not done via a "Plugin" but rather a "CPlugin" (as called in the source code files) or as we call it in the user interface "Controller".

I have not used the library you mentioned.
I only have implemented support for RN2483 as that module already has a fully certified LoRaWAN stack on board.

Typically those cheap "LoRa" modules out there only have a SX12xx chip for the LoRa radio transceiver.
But that's not yet LoRaWAN.
For LoRaWAN you need an additional LoRaWAN stack.
It is really hard to pick a proper software LoRaWAN stack which will be compliant with the LoRaWAN specifications.
For example when looking at "the" LMIC stack... There are so many variations, forks etc. of LMIC, you can't really speak about "the" LMIC stack.

So that's the reason I started with those RN2483 modules as it already is quite a steep learning curve to get started with LoRaWAN, thus eliminating extra unknowns sounded like a good start to me.

About the ESP8266 vs ESP32 part...
You can use ESP8266, but I would certainly advice against it for new designs as the amount of RAM and the rather limited max. sketch size are really a limiting factor which will bite you over and over again.
I think your own time is more valuable than a few dozen ESP8266 modules.


I strongly advice you to look into LoRaWAN a bit more before starting such an ambituous project.
It is for sure a really nice technique with lots of possibilities not seen in many other communication protocols. (only NB-IoT can be considered an alternative, as long as you're willing to pay a fee to a provider)

One thing about LoRaWAN that people often don't realize is that you can't send a packet from a gateway to a node.
You can only queue downlink messages which will be sent by the gateway as soon as an uplink packet is received by this gateway.
So if you only send out sample data every 15 minutes as a LoRaWAN uplink message then you might only send something to such a node as downlink within the RX-window after this uplink has been received.

Also you can only transmit for 1% of the time.
The "Spreading Factor" can be seen as some kind of indicator on how much time it may take to send out your data.
Let's assume sending some data packet at SF7 takes 100 msec to send.
At SF8 this will take 200 msec, at SF9 it takes 400 msec, etc.
Every step up, the time needed doubles.
And in this time you really can't send much data.
Typically your message should be less than 50-ish bytes.

This is just a very very short primer on all there is to know about LoRaWAN, so maybe you understand why I call it a "steep learning curve"
This has all been managed in the LoRaWAN RN2483 controller.
But this one only supports sending data, not yet receiving downlink messages.

[TD-er]

Nope, it is only supported to run as a node and only to send uplinks.
Support for downlinks isn't added yet.

[michcfr]

Thank you TD-er for the information.
Any chance to get it as LORA gateway soon?

[TD-er]

Nope, I am not planning on adding LoRa/LoRaWAN gateway/packet broker functionality to ESPEasy

You could have a look at the TTN Indoor Gateway which isn't that expensive.
But that one is tailored specifically for using TTN.