IOT framework, connected devices, device management
IoT development is about connected things – devices, vehicles, smart farms, buildings, wearables etc. that can transfer data over internet. In our IOT platform itself, we use different kinds of sensors which measures temperature, humidity, movement etc. They are attached to various assets and send these data every minute to Ripples IOT platform. This data is stored and analyzed to make appropriate decisions. Considering the diversity, scale and velocity of IoT data we require very smart database technology.
Traditional SQL databases are powerful in their querying and analyzing capabilities but the rate at which data can be inserted into DB may not be enough for IoT scenario. NoSQL databases have very high ingest rate and usually does not require strict schema. But they too come with limitations in analytical processing. Sometimes, we require a combination of various database technologies to arrive at a solution.
In our earlier IoT framework & integrated solutions we had only used MySQL as the database. We used its analytical capabilities in the form of powerful queries, procedures etc to arrive at decisions, create reporting etc. But for RipplesIOT, we expect a huge number of connected objects, so we have used a combination of the following database technologies:
– Cloudant NoSQL database/Mongo DB
– MySQL database
– DASH DB(on IBM cloud)
Cloudant NoSQL database & Mongo DB
In Ripples IOT framework, we process and enrich data from assets through node-red and store it as JSON in Cloudant in case of cloud solution(on IBM cloud) or MongoDB in case of local solution.
Cloudant is a distributed NoSQL database as a service (DBaaS) built from the ground up to scale and run non-stop. We can accessed Cloudant via Apache couchDB based JSON Restful API. Along with basic functionalities of a document oriented database like – create, read, update and delete, Cloudant also supports real-time map reduce, Apache Lucene full-text search and Cloudant Query.
We have used Cloudant Query to retrieve recent data to show in asset efficiency tracking application dashboard. We have used real-time map-reduce to aggregate daily data.
MongoDB is a popular free and open-source cross-platform document-oriented database. Data is stored as binary JSON in Mongo. Mongo DB supports powerful ad-hoc queries, indexing etc and there are provisions for load balancing, replication etc. We have used its powerful querying capabilities for showing data in dashboard as well as for aggregation.
MySQL database & IOT framework
Our IOT platform is based on Acquia Drupal 8, which uses the free and open-source relational database management system and MySQL at its backend. MySQL is cross-platform, SQL based and easy to use. It supports ACID transactions(atomicity, consistency, isolation, durability), procedures, triggers, indexing, failover etc. In local IOT platform, apart from Drupal data, we also use MySQL to store aggregated IoT data from assets, and each asset’s current location, usage state etc.
DASH DB & IOT development
In an IBM cloud based Asset monitoring system environment, we store aggregated data in DASH DB. DASH DB is IBM’s distributed cloud data warehouse. It supports inbuilt R analytics allowing us to use R scripts for analysis on data in storage. We store aggregated IoT sensor data from assets in tables here. We call R scripts on this data and perform basic predictive analytics functions.