RI Blog

Weather Station – server side application

rignSmart-Home

Hello. Today I would like to share with you next piece of my Weather Station project. It will be server side application which is responsible for aggregate data and serve it. As I mentioned in Weather Station – v3.0.0 my solution works with Home Assistant and with my own server application at the same time.

The whole project is based on popular Server Side framework NestJS which has very simple module structure and let create quick and easy applications. It also has additional benefit, especially for those who daily bases work with Angular that structure of modules, services, controlers and DI is similar to those known from Angular.

Code of my project is available on Github.

So let’s start from begin.

Database

All data from Weather Stations are stored in database. As you probably know there is a lot of different database systems, but in my project I use MySQL (I choose it because I know it, but you can use any other database).

Database schema contains three tables:

  • entity – list of object existing in my whole project (now is only Weather Station, but in the feature … who knows what can be stored here)
  • weather_station – basic information about Weather Station
  • weather_station_data – store data about temperature and humidity of all Weather Stations.

Entity

CREATE TABLE `entity` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `name` varchar(50) NOT NULL DEFAULT '',
  `ip` varchar(15) NOT NULL,
  `host` varchar(255) DEFAULT NULL,
  `topic` varchar(255) DEFAULT NULL,
  `topicSensorFull` varchar(255) DEFAULT NULL,
  `uniqId` varchar(255) DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `unqi_id_idx` (`uniqId`)
) ENGINE=InnoDB AUTO_INCREMENT=7 DEFAULT CHARSET=utf8;

Columns:

  • name – name of device
  • ip – current IP address
  • uniqId – unique ID of the device

Other columns are currently used by my oldest Weather Stations which use Tasmota firmware and before fully change to my software I need to have its. I suppose that in the nearest feature I will remove it.

Weather Station

CREATE TABLE `weather_station` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `name` varchar(50) NOT NULL DEFAULT '',
  `lastDataId` int(11) DEFAULT NULL,
  `entityId` int(11) DEFAULT NULL,
  `sensor` varchar(50) DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `REL_01e6c0501a3c932ddb7a91f161` (`lastDataId`),
  KEY `entity_idx` (`entityId`,`sensor`),
  CONSTRAINT `FK_01e6c0501a3c932ddb7a91f1618` FOREIGN KEY (`lastDataId`) REFERENCES `weather_station_data` (`id`) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB AUTO_INCREMENT=10 DEFAULT CHARSET=utf8;

Columns:

  • name – name of Weather Station for one Entity (you can have two different WS for one device)
  • lastDataId – newest ID record of temperature and humidity
  • sensor – symbol of the WS (0 – first WS, 1 – second WS)

Weather Station Data

CREATE TABLE `weather_station_data` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `temperature` double(4,2) NOT NULL,
  `humidity` double(4,2) NOT NULL,
  `weatherStationId` int(11) DEFAULT NULL,
  `timestamp` int(11) NOT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `timestamp_station` (`timestamp`,`weatherStationId`),
  KEY `FK_259d134eda7bdc7f07f72267475` (`weatherStationId`),
  CONSTRAINT `FK_259d134eda7bdc7f07f72267475` FOREIGN KEY (`weatherStationId`) REFERENCES `weather_station` (`id`) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB AUTO_INCREMENT=40824 DEFAULT CHARSET=utf8;

The last table contains all data for Weather Station:

  • temperature – temperature in oC
  • humidity – humidity in %
  • timestamp – timestamp of the measure
  • weatherStationId – Weather Station ID

I suppose that general concept of the database schema is clear and we can go to the process of saving data.

MQTT – collecting data

As I mentioned in Weather Station – v3.0.0, my Weather Station devices can communicate with server into two different ways, using HTTP or MQTT protocol. Each of that protocol has advantages and disadvantages, but from that point of my project I decided to use only MQTT. Because this protocol is more flexible and does not force me to send data twice to different servers.

My mechanism of collecting data is made from two events:

  • sending welcome message from the device
  • sending temperature and humidity data

Sending welcome message from the device

Generally we create applications and IT systems to make our life easy and to automate some repeating processes as much as it is possible. Because of that I decided to make fully integration between my Weather Station and server. As you know from my previous articles my Weather Station after turning on send “welcome message” on topic “ws/XXXXXX_Y/INFO” with message: 

{"ip":"192.168.1.2","uniqId":"999999","sensors":[{"symbol":0,"name":"11"},{"symbol":1,"name":"22"}]}

This such solutions allows me to check if I have registered device of such uniqId and I can update its data (how many sensor are connected to it and what kind of IP address it has) or create new one. This approach do not force me to set static IP address for each device or even manually init the device in database or in application.

It is worth to mention that such “welcome message” is not available in HTTP protocol.

Saving data

The next event in that application is listening on incoming sensor data. 

{"uniqId":"08A3A9","sensor":0,"payload": {"time":1596545956,"temp":"24.10","hum":"60.00"}}

The main goal of that listener is to save data of single Weather Station. In the message we send device uniqId and sensor number which points the device clearly. The incoming data could be archive or new. If it is new, then we set this measure as the newest measure for the device.

API server

The last part of my server solution is REST API application which can serve data different type of frontend application.

We can make few requests:

  • /api/weather-stations (GET) – return list of all Weather Stations registered in system
  • /api/weather-stations/:ID/data?from=FROM_TIMESTAMP&to=TO_TIMESTAMP (GET) – return average temperature and humidity for every day between passing timestamps (FROM_TIMESTAM and TO_TIMESTAMP is value in milliseconds)
  • /api/weather-stations/:ID/data/month?from=YEAR&month=MONTH – return average temperature and humidity for every day for YEAR and MONTH and weather station ID (month has numbers from 0 to 11)
  • /api/weather-stations/:ID/data/year?from=YEAR – return average temperature and humidity for every month for YEAR and weather station ID
  • /api/weather-stations/:ID/data/week?from=YEAR&month=MONTH&day=DAY – return average temperature and humidity for every day of the WEEK starting from YEAR-MONTH-DAY for weather station ID (month has numbers from 0 to 11)
  • /api/weather-stations/:ID/data/day?from=YEAR&month=MONTH&day=DAY – return average temperature and humidity for every HOUR for the passing day YEAR-MONTH-DAY and weather station ID (month has numbers from 0 to 11)

There is also one more request which is only use when Weather Station use HTTP protocol.

  • /api/weather-stations/sync (POST) – save weather station sensor data
{ 
  "ip": "10.10.10.11", 
  "sensor": "0", 
  "data": [
    {
      "time": timestamp_in_seconds,
      "temp": "24.14",
      "hum": "80.92"
    },
    ...
  ]
}

This time System based on IP address and sensor parameter save that for proper Weather Station device.

Static files

Additional functionality of API application is to server static files. 

//main.ts
...
async function bootstrap() {
  ...
  app.useStaticAssets(join(__dirname, 'app'));

  ...
}

bootstrap();

Because of such solutions I can serve static files for app directory as it will be serve from root. I use that approach to serve my compiled Angular application which presents all data using above API.

When I run this server application I have to paths:

  • / – serving static files (Angular app)
  • /api – path that contains above API request

Summary

Generally that is all. I could go deep and deep into code of my application, but in that article I would like to focus on main concept and responsibility of each part of application. Everyone could ask now, how do you run it? The answer is very simple – Docker on Raspberry Pi 4. How I configure it? How look like the frontend app? I will answer on all that questions in my next article.

Weather Station – server side application
Scroll to top