Datastringer will help you subscribe to data sources, and will notify you when patterns arise or thresholds are broken.
It is more or less the exact equivalent of wire agencies’ (AFP/AP/Reuters) local journalists who feed the organisation with news (called wires). Except that here, that’s a software living in a dataset, not in Iran, that’s going to do the job.
Picture yourself as a local journalist. You love local police stories, there’s always something to write about, and your audience cares a lot about them. You get these stories by calling your sources in the force in the morning, and by receiving press releases every once in a while. How fantastic would it be if you could have a little piece of software you could tell what you’re looking for, then let it do the leg-work and alert you when it finds something worth your attention?
That’s what Datastringer aims to do. It was built to be a simple as possible, and very much journalism-oriented. The idea is not new in itself, it’s basically a monitoring tool. However, Datastringer wants to:
- Simplify the process of monitoring streaming pure-data sources,
- Provide a ready-to-use environment to help you define your alert parameters, as well as encourage you to perform basic data-analysis operations on the monitored datasets before setting the alerts.
See the Datastringer Documentation and explanation here for further info.
What is DataStringer?
As journalists, we’re spending an awful amount of time reading the news. Going through the papers, our favourite news websites, our Twitter lists, and our RSS feeds, is part of our morning routine. At work, the familiar ticking of wires coming in never stops.
Some of us now work with data too, considering it a source in itself; a source which can give us tips, leads, and stories, just like our informant working in the Home office.
Summary: we designed Datastringer in an easy to understand way, so you can hopefully grasp the concept quickly… and get started with your own stringers!
datastringer.js is the main program. It uses two concepts:
In Datastringer, each stringer is stored in its own js file, named after what the stringer is about. It must be located at the root of the repository.
Datastringer comes with two builtin stringers, located in
You can write your own stringers too! This page will get you started.
Datastringer stores the use-cases in
~/.local/share/datastringer/stringer_use_cases.json (when you first clone the repo, the file is’nt there: you can write it from scratch or run the wizard, which generate one for you). If you peak inside it you’ll see something like that:
The use cases file is very easy to tweak and extend. Go over there to see how you can do that!
The main program just reads the list of use-cases stored in
~/.local/share/datastringer/stringers_use_cases.json, and for each one the
concerned stringer is run with the given set of parameters, plus a callback that
will send you a mail if it is called. And that’s about it!
Again, note that this file is not present by default when you clone the repo. It will be created when you run the wizard for the first time.