NicTheGeek

#lora

Store your TTN node data in a NoSQL database, then show it as a line graph on a webpage…

This blog post is all about building a Node.js server to retrieve your LoRaWAN device data from TheThingsNetwork (TTN), then store the data in a Mongo NoSQL database. The assumption is that you have built a Node.js app before. There is also a very simple client app that displays your data as a line graph on a webpage.

The Things Network (TTN) is an open-source, secure and scalable solution that has around 9,000 gateways and 90,000 developers around the world. It started in August 2015 and just keeps on growing.

TTN manages every aspect of creating and managing a LoRaWAN network except storing your data (note that TTN does offer a storage solution, but provides storage for 7 days only). Also note that there are several integrations available, including EVRYTHNG, AWS IoT, Cayenne and Tago.

The following example is useful if you want to keep your data in-house. Also, if you are like me, you want to understand and have control of the key components of your solution, even if you decide in the future to go with a 3rd-party.

This code runs as two separate Node.js applications on your laptop/desktop, but of course when moving to production, the collector app would need to run on a server and the viewer app is actually a web server that would also need to run continuously.

In addition to the code described below, you need:
Node.js – javascript server-side run-time environment – I used version 10.16.3 LTS
MongoDB – NoSQL database, ideal for json-formatted data – I used a free cloud instance, sign up here (no credit card required) https://www.mongodb.com/cloud/atlas – and of course an active LoRaWAN device, registered within an application configured in your own TTN console – https://console.thethingsnetwork.org/

You can find all the source code, together with the slidedeck I presented at TheThingsConference UK 2019, here:

https://github.com/nicbkw/thingsconf2019

I have split the code into collector and viewer folders. You need to fill out the connection details for your mongodb  instance and your TTN application name and access key (token). Use the config.template as a basis, update with your credentials and then place a copy, called config.js,  into each of the collector and viewer folders on your laptop/desktop.

First, let’s take a look at collector; this uses a Node.js SDK provided by TTN that listens for each new published record, then inserts a document into your database. You can see the code for this in collector.js – note that I have created a separate file, mongo.js, for mongodb connectivity, as this is common code for both collector and viewer.

A quick diversion to talk about nomenclature – both SQL and NoSQL databases use the term ‘database’ – but a ‘table’ in SQL is equivalent to a ‘collection’ in NoSQL and a ‘record’ in SQL is a ‘document’ in NoSQL. What I really like about NoSQL databases is that the underlying format is JSON.

You can check the data is being stored by installing mongodb compass, community edition – https://www.mongodb.com/download-center/compass

OK, so now that we have data in our database, let’s view it in a webpage as a line graph. The viewer folder holds a copy of config.js (note that this only needs to hold the credentials for the database) a copy of mongo.js for database connectivity, the application code in viewer.js and a sub-folder called views that holds index.hbs – a simplified html file.

The Node.js app actually runs the minimum form of express framework to host a simple web page. Our web page is served using ‘hbs’ as it’s default view engine (https://www.npmjs.com/package/hbs) and we use handlebar nomenclature {{JSON string}} to pass data arrays. So, take a look at viewer.js and see that we retrieve a number of documents from the database with ‘getArray’ and then push the relevant data into an array in ‘pushData’ to pass to the web page where we render it. We are going to plot the ‘rssi’ and ‘snr’ as seen by the first gateway to respond to your LoRaWAN device.

The file index.hbs contains the minimal html and javascript to display the data as two line graphs, using the Google charts libraries – more information here: https://developers.google.com/chart

So, in summary:
TTN provides an excellent platform to move your node data to the cloud
TTN enables simple data access with a Node.js SDK
MongoDB is a NoSQL db, good for JSON data
Node.js is a simple way to build a headless server app

Now, it’s down to you to do something useful with your LoRaWAN device!

Tags: , , , , ,

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Technical overview of LoRa, LoRaWAN and The Things Network

The Radio Spectrum

Radio-connected devices work within a certain frequency range. The radio spectrum is allocated by government and licensed for specific functions.

For example, broadcast radio currently has a reserved VHF band from 88 to 108 MHz.

Cellular phones have several reserved frequency bands around 900 MHz and 1800 MHz.

The licensees pay for use within these bands. However, there are some specific bands that are made available for low power use, license-free.

2.4GHz is a global band that allows low power unlicensed operation; this is where WiFi and Bluetooth co-exist.

Some license-free bands are allocated differently by each country and LoRa is designed to work in the appropriate band for each country or region. For example, Europe is covered by 863-870MHz, North America, Canada & South America by 902-928MHz. TTN provide details of global frequency plans – https://www.thethingsnetwork.org/docs/lorawan/frequency-plans.html

LoRa radio technology

LoRa is a technology that works well within a crowded radio spectrum.

It uses a technique called chirp spread spectrum that creates a signal which is distinguishable by the receiver from the atmospheric noise and signals created by other devices that use simpler modulation techniques, such as On Off Keying (OOK) and Frequency Shift Keying (FSK).

LoRa also takes advantage of a feature called Spreading Factor (SF) where the spreading factors are the duration of the chirp. LoRa operates with SF7-12. SF7 is the shortest time on air, SF12 the longest. Each step up in spreading factor doubles the time on air to transmit the same amount of data.

With the same bandwidth, longer time on air obviously results in less data transmitted per unit of time.

So, with very little output power, the LoRa signal can be detected at great distance or through infrastructure that attenuates the signal. Typically, this could be 1-3km in an urban environment. The current record distance is just over 700km from a node under a helium balloon at a height of 40km, at an output power of only 25mw. For comparison your car key fob works at 2mw for a couple of metres and your phone at up to 1W to the nearest cellphone mast, at most 30km distant.

The trade-off for LoRa is that low power and long distance is traded for limited data rate. Maximum data rate at SF7 is 6kbits/s, reducing to 300bits/s at SF12. One key figure RF engineers discuss is “link budget”, bigger is better and Spreading Factor affects the number where SF12 gives the best chance to receive data. For more information – https://www.semtech.com/uploads/documents/an1200.22.pdf

There are payload size limitations for LoRa – in EU it’s 230 bytes per transmission at SF7, 59 bytes at SF12. Consider LoRa only for telemetry data, such as location, heading, height, temperature, pressure and humidity values or simple on-off actions.

The LoRaWAN protocol

LoRaWAN is a communication protocol that runs on LoRa hardware.

LoRaWAN operates in a star of stars network with ability for multiple gateways to receive and forward data from any nearby node to The Things Network.

Each multi-channel LoRaWAN gateway can scan 8 channels simultaneously and decode up to 8 data packets at the same time. Several packets using different data rates may be demodulated simultaneously even on the same channel.

LoRaWAN provides several modes of operation – modes A, B & C, where B is allocated a time slot and C devices are permanently powered on and receiving.

Most nodes run as mode A devices where they occasionally wake up to transmit a small amount of data and listen for a short time after transmission for any received data. Working like this, it is possible to construct a node that can transmit an update every 30 minutes and run off 2 ‘C’-size alkaline batteries for more than 5 years.

Each mode A node is free to transmit at any time, with the limitation in Europe that each node should not exceed a 1% duty cycle.

There are predictive formulae available to show that the combination of multi-channel gateways, small data packets, operational modes and Spreading Factors results in the capability to handle thousands of nodes per gateway. See “Understanding the Limits of LoRaWAN (Adelantado, Vilajosana et al, IEEE magazine January 2017)” – https://arxiv.org/pdf/1607.08011.pdf

The Things Network originally provided complete LoRaWAN coverage for the city of Amsterdam with just 10 gateways – https://www.thethingsnetwork.org/community/amsterdam/

The Things Network

The Things Network is an open-source, free-to-use solution that provides all functions necessary to:

  • Register a gateway
  • Register a node
  • Handle encryption and decryption of payload data.
  • Route the data between node and the internet using MQTT-broker technology.
  • Queue data for transmission to each node.
  • Monitor and action the process through API

The Things Network does all this and more on a scalable, resilient platform with regional bridge access globally.

https://www.thethingsnetwork.org/docs/network/architecture.html

The Things Network is provided as open-source solution by The Things Industries. Gateways are being made available worldwide by The Things Network community – see https://www.thethingsnetwork.org/community and try it for yourself! it costs less than £200 to setup your own.

The Things Industries also deliver private LoRaWAN solutions with guaranteed SLA, so be assured that you can develop a solution and build it out to a stable product.

For more information, please get in touch – take a look at https://thinnovation.com/remon.html

Tags: , , ,

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.