Scene Me

Video analytics often mean acting on some pre-defined event producing analytics.  What if you can get hold of the underlaying analytics to process and create your own specific events, logs or visualization?

Scene Me replaces my previous ACAP MQTT Me and Radar Me (they are no longer maintained).  Scene Me is an extension of MQTT Me with the additional support of publishing analytics data. The analytics in this case are trackers, objects and scenes.   Trackers is the data of moving objects within the camera view.  Tracker data is published continuously while the video object moves across the scene.  Trackers can be used for real-time event filters detecting specific behavior or visualization/augmentation.  When the tracker loses the video object an “object” is published.  An object is a data structure that includes place of birth/death and time of birth/death.   While a tracker targets real-time processing, objects target post-processing applications that have no need to process trackers.  The third type, “scene” is a data structured that is published when a tracker fulfils a specific behavior filter during the time a tracker is active.  A scene is a filter of place-of-birth, minimum size, minimum age and minimum vertical and/or horizontal movement.  Scenes are “events”, similar to “motion detection” but having multiple properties, making it possible to detect specific behavior.  Scenes are also automatically published on the ONVIF event stream.

What about Integration?

MQTT is a publish/subscribe protocol where data is published with a topic and a payload.  MQTT clients do not connect directly to the device, they connects to a MQTT broker.  Multiple clients can subscribe to teh same topic can also publish messages.  The broker becomes the system data bus.  A client may say to the broker “send me all the messages with the topic “tracker/#”.  This will give the client all the trackers from all the cameras in a single subscription.  It does not need to connect to each end every camera.  It is a single-point-of-integration and at the same time de-coupling the publisher from the consumer.  The subscriber does not need to not have direct access to the device producing the data.

This integration requires a MQTT broker.  Installing and operating a broker is easy.  If you do not have any possibilities hosting a private mosquitto broker, you can use a public  enterprise MQTT broker service on Internet (CloudMQTT, HIVEMQ or other).

Does this integrate with my VMS?

To my knowledge there are no VMS that have a built-in MQTT client. I believe it is just a matter of time.  However, there are work-arounds to integrate MQTT messaging into a VMS depending on use-case.

Scene Me supports “scenes”.  A defined scene in Scene Me will be automatically sent as an “event” on the ONVIF event stream that most VMS support.  For this case you do not need an MQTT broker to make the integration.

Scene Me can subscribe to a user defined MQTT topic and generate an ONVIF event that a VMS can monitor.  This makes it possible for a 3:rd party MQTT client ( e.g. an Android/IOS  MQTT Dashboard) to trigger actions in the VMS, using a camera with Scene Me as a bridge/proxy.

Node-Red, an integrators dream

A VMS is a Video Management System.  Counting objects, visual augementation, controlling lights, sending alerts, optimizing building/business automation systems may not require a VMS.  What is needed is a Data Management Service that can integrate and process data from and to a VMS, other devices or other systemes.  As long as devices and systems have  some public API you can integrate.

Out of all the different ways of doing integration, including C, PHP, Python and more, I have found Node-Red to be the easiest and most dynamic tool out there.  Originally developed by IBM’s Emerging Technology Services, this web-based flow editor makes it easy to wire togheter inputs and outputs including MQTT, HTTP, TCP, DataBase, files, timers, email and more.  Knowing some basic Node/JavaScript is needed for complex logic but the vast amount of community flows and nodes allows you to do a lot without any programming skills. You can also add a user dashboard where users can view data/graphs or initiate actions to the systems. Node-Red can be hosted on a Rasberry Pi, Windows server, Linux Server or VM.  It is free of charge.  Download and play!  My own system runs a Node-Red server and a Mosquitto broker on the same machine that hosts the VMS.  Adding a Mongo DataBase completes the system.  My Node-Red integrates to my VMS, my home automation system, my SONOS (voice notifications) and my mobile phones (GPS tracking, Zones and notifications).

As an example, the following screenshot shows how Node-Red sends a Scene Me Video analytics object to a database, email, twitter, file and http at the same time.

References/Links

Documentation

The user documentation [PDF] describes the overview of Scene Me.

License

As always, this is a freeware ACAP service but license registration.  License registrations requires an “Axis MyAccount”.  Go to Axis Portal for ACAP license registation, register the serial number of your camera and select “Free application” and search for “Scene Me”.  Download and install the license key file in you Axis Device.

Supported Axis Devices

All Axis devices based on MIPS or ARM7hf with firmware 6.50 or later.  In order to utilize video analystics, the firmware must be 9.20 or later.

Note:  If you have previously installed MQTT Me, remove that ACAP before starting Scene Me.  You may also need to adjust your integration as some topics have changed.

Download Scene Me

Release notes [TXT]

 

2 comments

  1. Hello, this is a very nice tool indeed.

    Is the source code for this available? I wonder why supporting TLS is hard. For instance, MQTT-C is a 2k lines C library that supports TLS and only depends on C standard libraries. What are you using instead of that? Thanks.

    Like

    1. Sorry, I do not share any source code. I used a cross-compiled MQTT lib and did not include the lib for TLS as I did not not have the needs. The messages are not critical and TCP is sufficient for local LAN broker. If needed to publish to public broker you can republish messages over TLS from a local broker or use scripting like Node-Red.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s