Why I switched from OpenTSDB to KairosDB?

In my previous post, I described how to correctly install and use OpenTSDB. After some time, I decided to move on to other solution.

The story

Before everything, we need to know one thing. Because of IoT, the demand for storing sensor data has increased dramatically. Many new projects emerged, some are good, some are bad. They are different in technologies used, how fast they are and what kind of features they support.

You can read the full list of all IoT timeseries databases that can be used for storing data of you Internet of Things projects or startup.

Problems of OpenTSDB

OpenTSDB is great, don’t get me wrong. But when you try to use is with some more complex projects and customer demands, you can quickly hit the wall. It’s mostly because it involves a lot of moving parts to make it work (Hadoop, HBase, ZooKeeper). If one of the parts fail, the whole thing fails. Sure, you can replicate each thing and make it more robust, but you will also spend more money. When you are starting, it’s a over optimization and waste of money (that you don’t have).

Aggregation of the data is another problem. It does support basic function like min, max, avg etc. I spent days investigating the problem why avg aggregation is not working correctly when I filter by multiple tags. It just didn’t want to work and I couldn’t find anything in the docs. I asked on Google group and after some time I got a reply that I must use another aggregation function and that even that doesn’t work 100% as I want it. Another problem is when I want to get just one value – for example avg of all values from X to now. Not possible!

No clients to talk with OpenTSDB is another problem for me. Sure, storing the data with socket API is super simple and can be easily integrated in every language. The HTTP API is another story. Sure, again it shouldn’t be a problem to implement my own client, but why waste time with this?

Development of the OpenTSDB is slow and it takes ages for new features to be integrated. One of them (one of the most important for me) is an ability to support time zones. It’s used when downsampling data to one day (or even more) so data is correctly grouped. There was some work, but until today it still wasn’t implemented. Too bad.

On the bright side, OpenTSDB is super fast. I was able to store and load data as super fast rate – loading 3 million records in few seconds is for me super fast. Try it with relational database and you will be quickly disappointed.

KairosDB to the rescue

I remember when I was doing a research, I noticed KairosDB but I didn’t spend too much time testing it. It just wasn’t appealing and I didn’t know how it actually works. Big mistake.

KairosDB uses Cassandra to store data (compared to HBase used with OpenTSDB) and it’s actually a rewritten and upgraded version of OpenTSDB. It has evolved into great project. It has many more features: many more (and fully working) aggregation methods, option to easily delete metric or datapoint, easy extensibility with plugins etc. It has great clients and has much more active community. I remember when I asked a question on OpenTSDB Google group and waited weeks for an answer (I’m not forcing anyone to provide the support, because after all, it’s an opensource project), while on KairosDB Google group I got it within a day.

Why is this important you might ask? Well, when you are catching deadlines and something goes wrong, responsive community is very important. Sometimes this kind of things can be a difference between success and a failure.

What now?

I wrote an tutorial how to start with KairosDB. You can also you visit kairosdb.org and check out the documentation. Feel free to play with it, test it and hopefully also use it in production. I

Best Internet of Things books

First we need to learn few things about IoT. Although the concept wasn’t named until 1999, the Internet of Things has been in development for decades. The first Internet appliance, for example, was a Coke machine at Carnegie Melon University in the early 1980s. The programmers could connect to the machine over the Internet, check the status of the machine and determine whether or not there would be a cold drink awaiting them, should they decide to make the trip down to the machine.

Kevin Ashton, cofounder and executive director of the Auto-ID Center at MIT, first mentioned the Internet of Things in a presentation he made to Procter & Gamble. From that day, IoT started to slowly grow and today it’s one of the fastest growing trends.

10 best Internet of Things books

One of the best ways to learn about IoT is using books. I have completed the list of some of the best and a must reads.

  1. Internet of Things (A Hands-on-Approach) (2014)
    Internet of Things (IoT) refers to physical and virtual objects that have unique identities and are connected to the internet to facilitate intelligent applications that make energy, logistics, industrial control, retail, agriculture and many other domains “smarter”.
  2. Designing the Internet of Things (2013)
    Whether it’s called physical computing, ubiquitous computing, or the Internet of Things, it’s a hot topic in technology: how to channel your inner Steve Jobs and successfully combine hardware, embedded software, web services, electronics, and cool design to create cutting-edge devices that are fun, interactive, and practical. If you’d like to create the next must-have product, this unique book is the perfect place to start.
  3. The Internet of Things (2014)
    You might only have heard this expression recently (indeed you might never had heard of it) but, apparently this is a concept that has been around for some time.The term was coined around the turn of the millennium and refers to the potential interconnectivity of basically all electronic devices and capacity to record, monitor and transmit information between them to achieve all manner of wonderful (and maybe not-so-wonderful) outcomes.
  4. The Silent Intelligence: The Internet of Things (2013)
    The Silent Intelligence is a book about the Internet of Things. We talk about the history, trends, technology ecosystem and future of Connected Cities, Connected Homes, Connected Health and Connected Cars. We also discuss the most exciting growth areas for entrepreneurs and venture capital investors.
  5. Rethinking the Internet of Things: A Scalable Approach to Connecting Everything (2013)
    Over the next decade, most devices connected to the Internet will not be used by people in the familiar way that personal computers, tablets and smart phones are. Billions of interconnected devices will be monitoring the environment, transportation systems, factories, farms, forests, utilities, soil and weather conditions, oceans and resources.
  6. The Epic Struggle of the Internet of Things (2014)
    If the hype is to be believed then the next big thing is the Internet of Things. But is it what you think it is? Because the Internet of Things is not about things on the internet. A world in which all our household gadgets can communicate with each other may sound vaguely useful, but it’s not really for us consumers.
  7. Designing for Emerging Technologies: UX for Genomics, Robotics, and the Internet of Things (2014)
    The recent digital and mobile revolutions are a minor blip compared to the next wave of technological change, as everything from robot swarms to skin-top embeddable computers and bio printable organs start appearing in coming years. In this collection of inspiring essays, designers, engineers, and researchers discuss their approaches to experience design for groundbreaking technologies.
  8. From Machine-to-Machine to the Internet of Things: Introduction to a New Age of Intelligence (2014)
    This book outlines the background and overall vision for the Internet of Things (IoT) and Machine-to-Machine (M2M) communications and services, including major standards. Key technologies are described, and include everything from physical instrumentation of devices to the cloud infrastructures used to collect data.
  9. Enchanted Objects: Design, Human Desire, and the Internet of Things (2014)
    In the tradition of Who Owns the Future? and The Second Machine Age, an MIT Media Lab scientist imagines how everyday objects can intuit our needs and improve our lives. We are now standing at the precipice of the next transformative development: the Internet of Things.
  10. Invest, Make Money & Retire Early From The Internet Of Things Revolution (2014)
    The internet of things will equate to one of the single most lucrative investment opportunities in the history of modern business and will be bigger than the economies of China, Norway, and Canada, combined. Simply put the internet of things is connecting any device with an on and off switch to the Internet (and/or to each other).

9 websites to find a co-founder/partner for your startup

Finding the right person to work with you on your project or product is really hard. One way is to ask around, you friends and family, and see if they can recommend someone. Mostly this way brings bad results. Other, in some cases, much better way is to use different sites to find your business partner.

I have created a short list of websites and short description from their website.

1. CoFoundersLab (more than 25000 users)

Join the largest community of more than 25,000 founders, advisers, and interns to help launch or grow your business.

2. Founder2be (more than 21000 users)

Founder2be is the largest startup community for developers, web designers, marketers and anyone looking to start a startup worldwide.

3. Founders Nation

Founders Nation is a platform for entrepreneurs and founders. We built it to connect dreamers that wish to make the world a better place through innovation and technology.

4. FounderDating

FounderDating is a network of talented entrepreneurs helping one another start and grow companies. With FounderDating you find world-class people with complimentary skill-sets, expert areas and knowledge.

5. CollabFinder

Find cofounders, collaborators and other makers to help build your project. CollabFinder Groups give your community members a place to team up and launch projects.

6. Bizoogo

Where UK entrepreneurs connect, swap, share and collaborate to start-up.

7. cofinder.eu (creators are from Slovenia)

Helping you assemble A STARTUP TEAM.

8. FoundersHookup

FoundersHookup’s goal is to connect you with GREAT people to seed high quality DNA into your venture from Day One. We screen applicants and only present the highest caliber Internet founder prospects.

9. Reddit, section /r/cofounder

Even though reddit is use for many things, if also offers a section for finding a cofounder. It’s not so great as previous websites, but maybe someone can find a dream match.

If you think I skipped an important website to find a co-founder, send me an email and I will update the list.

OpenTSDB – the perfect database for your Internet of Things projects

I got a call the other day with a question: how can we store huge amount of sensor data. They are measuring air temperature in different rooms every 5 seconds. That means 17280 data points per data, 6307200 data points per year and for 15 rooms 94608000 data points per year.

Because I never had a situation where I needed to store a huge amount of sensor data, I didn’t know the answer. But I started digging. There are many questions online regarding what database to use to store this kind of data. Some recommend old-school databases like MySQL or Oracle. Some recommend Redis, Riak or MongoDB. But one recommendation beat them all: OpenTSDB.

OpenTSDB – The Scalable Time Series Database
Store and server massive amounts of time series data without losing granularity.

Currently in version 2.0, OpenTSDB is a tested solution build on top of HBase. It was designed especially for time series data and can handle

– up to 100+ billion data points and
– 2000 new data points per second (tested by OpenTSDB developers on a old dual-core Intel Xeon CPU from 2006; I tested on a newer machine and could easily insert 20000 points in few seconds).

Long story short. It’s perfect database for huge amount of sensor data. It has great options to query data (I will explain it below), has additional features to annotate data and it’s under active development.

Installation and running it for the first time

To run OpenTSDB, you need to first install HBase. The procedure is pretty straightforward. First you need to install HBase. Download HBase, unpack, define configuration and run it with

If everything was defined correctly, you should get a message

Next step is installing OpenTSDB. There is a great tutorial how to install OpenTSDB. In short, download it and unpack or clone git repository, and run build.

It should take few minutes to compile everything. Next step is to create tables with command

You can see created tables with few opensource HBase viewers like hrider. Currently the compression is set to none. It’s highly recommend to use compression LZO, because there is no performance impact but it can highly reduce the size of your data.

Because we will store temperatures in metric temperatures, we need to create it first. OpenTSDB has a configuration to enable auto creation of metrics, but it’s not recommended, so we will do it manually.

The last step is to run everything.

If everything went well, you should see OpenTSDB page at localhost:4242. It’s that simple.

How data is stored

How OpenTSDB is storing the data is in my opinion the biggest difference compared to other databases. It does support tables, but they are actually called metrics. In each metric we can store data points. Each data points is structured as

Timestamp (unix time or ISO 9601 format) is the time of the data point. Value is a number (integer or float). Then we have tags. With tags we separate data points. In our example, we are storing value for bedroom on our first floor. This structure enables us to separate data and later make advanced queries; for example average temperature on first floor or sum of all rooms.

Storing data

With version 2.0, OpenTSDB has 2 ways to store and access data (plus one additional to store by importing data). They are Telnet API, HTTP API and batch import from a file. Make sure you have OpenTSDB running before you try the examples below.

Storing with Telnet API [Java]

We need to execute command PUT with metric and data. še dopiši

Storing with HTTP API
When working with HTTP API, we have to make a PUT request to the URL localhost:4242/api/put with JSON data.

There is also a possibility to make a batch insert. Just wrap all metrics in an array.

Personally I had few problems inserting a large amount of data with the API. I ended up using Telnet API and it seems to work really well.

Querying the data

The whole beauty of OpenTSDB is it’s ability to not only to store huge amount of data, but to also query it fast. I will be showing how to query data with HTTP API, but the same query parameters can be used with Telnet API.

For the examples, we will first insert some data. Of course we can insert a much large dataset, but for this tutorial lets keep it simple.

Getting all temperatures

Let’s break down the request:
1. We can make GET or POST requests
2. The HTTP API URL is http://localhost:4242/api/query
3. We must define start, but end is optional. It can be unix timestamp or you can define nx-ago where n is unit and x is metric. For example, 1day-ago or 1h-ago. OpenTSDB will automatically convert it to timestamp based on your time.
4. m is the metric, where we are using aggregation = sum and metric = temperatures.
5. The last is grouping_operator (inside {}), which is used to group the data. If we define it with *, then it will not group the data. We can also use it to filter. For example room=bedroom will only fetch data from bedroom.

You can read more about different parameters and what they do at http://opentsdb.net/docs/build/html/api_http/query/index.html.

Our above request returns JSON.

Getting temperatures in the bedroom

As mentioned above, we can query by tags. In our case by room=bedroom.


Getting average temperature on first floor

To calculate the average of the temperatures on first floor, we have to group by tags. Be careful to define correct aggregation function (in our case avg). See all aggregators at http://opentsdb.net/docs/build/html/user_guide/query/aggregators.html#available-aggregators.


We can see tag room in aggregateTags. It means it used this tag to aggregate (or if you are familiar with other databases, GROUP BY) data.

Getting average temperatures per day

Let’s imagine a situation where we want to create reports of the temperatures on a daily basis. We could load all the data and then manually calculate the averages. For larger datasets it could take some time. OpenTSDB has an answer. We can also define downsampling. Downsampling will automatically calculate the values based on our downsampling aggregation function and timeframe.

Notice different parameter m? We added 1d-avg (be careful to separate everything correctly with “:”), which will downsample by 1 day and calculate average. Compared to manual way, it’s much faster and it just gives us results, which we can use in graphs.

Other awesome features

OpenTSDB has few additional features to cover real-life situations. Of course we can easily add more with plugins. But 2 of them worth mentioning are Annotations and CLI Tools.

Annotations enable us to add additional meta data to data points. For example, we could store information when we opened and closed window in each room or when we changed the heating level.

Read more at http://opentsdb.net/docs/build/html/api_http/annotation.html.

CLI Tools

CLI Tools are just simple tools to perform additional task like fixing the data storage (in case if something breaks down), querying and deleting data and creating metrics. One of the most common tools I use it scan, because it has the feature to delete data. t’s useful when we are doing different tests.

To delete all temperatures for basement, we execute command

Again, we can filter what to delete with start and end parameters, metric and tags.

Wrap up

OpenTSDB has been proved to be an excellent solution. It’s scalable, fast and has really neat features. Most importantly, it’s under active development and has many people contributing. With the era of IoT and Big Data upon us, it has a bright future ahead.

If you are ready, start with http://opentsdb.net/docs/build/html/index.html.

Hekovnik & start:Cloud – why me, why now and why should I f*cking care?

I have already been member of one startup Osmosis, the charity for 21st century. We won the well established ImagineCup competition in Slovenia in 2012 and finished 7th at world finals. It was an amazing experience (I would like to thank again to all that helped us) and after 1 year I still have live memories of our hard work and determination. But the focus was totally elsewhere. TAM, SAM, SOM, WHAT? Is that something I can code with? No. Then I don’t care. We were focused on our product and not how others see it and how they could use it. So when we finally approached the market, we realized that what we imagined will never work. Our knowledge of charity market was insufficient and our product didn’t fit. At the same time we ran out of steam so we decided to stop Osmosis.

There is nothing worse then to fail in Slovenia. People look at you strange and they see you as failure. As someone who doesn’t know what they are doing and will never succeed. Because if you don’t succeed for the first time, you never will. Period. I don’t think the same. I was determined to start over and try something else. This is where Hekovnik comes in. Hekovnik is, I quote

Hekovnik Startup School provides targeted accelerator courses to hackers, entrepreneurs and academia to help them change the world.

In other words. Hekovnik is a startup school which will guide you from your idea to your first product and customers and even beyond that. It’s a place where like-minded people hangout and share ideas and knowledge. It’s A MUST place if you are just starting and you need help or just want to meet new people.

But let’s not talk too abstract. Let me describe how Hekovnik, their workshops and the guys working there helped us with MEDinar. I’m will divide it in 3 sections: working space, programs & workshops and team.

1. Working space

It’s located in the old Technology Park Ljubljana (Teslova 30, 1000 Ljubljana), 1 minute from Faculty of Computer and Information Science. It offers 3 big rooms: Tearoom (Čajnica), Black hole (Črna luknja) and Storming room (Viharnica).  For us most important is Tearoom. We used it as our office for few months for free. With it’s many big white boards it’s perfect for brainstorming and working. At the same time, it’s the gathering point, so you can meet many great people and get answers to your questions. We sure did. The bottom line is. Don’t sit behind computer at home. Come in Hekovnik because it offers amazing work-space. Plus they have a fast Wi-Fi.

2. Programs & Workshops

Hekovnik offers many programs. I cannot talk about all of them, but from rumors I heard, they are amazing. I can talk about start:Cloud program, about our experience and how it works. When you signup (as a student we paid 50€) you get a book The Startup Owner’s Manual: The Step-By-Step Guide for Building a Great Company. I highly recommend it. Program is divided in weekly workshops (every Tuesday and Thursday). On the workshops, first there are lectures and then you get to work. Hekovnik follows Lean Startup Methodology and during workshops they will demonstrate different tools and how to use them to test your hypothesis, find you customer segment, build your MVP and so on. At first, we didn’t know where to start. With talking to people and reading articles we tried to learn as much as possible about medical industry. With workshops and their help we found our customer segment and what problems they have. We made simple landing page to test our hypothesis, started building a web channel to reach them, offered them our solution and on the end calculated how much it’s gonna cost us to develop it, and for how much we will sell it. We made small, but steady progress each and every day.

The idea when all the teams are in the same room, each working on their product, on the end presenting what they did, commenting and receiving comments from other teams is just AWESOME. I really mean it. Because ideas don’t bounce off blackboards, they bounce off people. This is my favorite advice from Kristjan. Getting the comments from others and learning from mistakes helps you advance much faster. Workshops are intense and packed with knowledge and information. On the end they give you the right tools and frameworks, enough knowledge and experiences, and most important of all: they prepare you for the real world.

3. Team

There are many experts in Slovenia. There are also many so-called experts. But I think based on experience and professional approach, Hekovnik team knows what it’s doing.

Kristjan is the visionary and leader with wast knowledge in building startups. He hosts most of the workshops and talks with passion.  I always remember him walking in Tearoom like Kramer from Seinfeld. They are about the same height and both have crazy ideas. But he always asked how we are and what we are doing. After we told him and described our problem, he gave us some ideas and pointers. Keeping someone like him near is very important to stay focused.

Next team member is Tomaž. I think his job is to make sure everything runs smoothly. He helped us with our questions regarding start:Cloud. He also helped us with business questions and connected us with the right people. You just have to ask him nicely. When we went to annual Geekend, I remember him telling us Silicon Valley stories around camp fire. I think he could write a book about them.

Third member is Filip. He has a lot of experience in presentations. Even though he is a head smaller and probably weights half my kilos, I was terrified of him. When we worked on presentation, slogan, short description or anything else meant for general public and presented it to him, he always “destroyed” us. I remember him saying: I only hear blablabla.  He doesn’t mean anything bad, but he really helped us to better describe and express our product and always pushed us to make it better.

There are also other members, Matic and Matic. One responsible that all information reached us, and other that all the technical things worked. He also made sure we were not hungry during workshops. All I can say, Subway who? Hekovnik sandwiches FTW!

Few days ago we finished our start:Cloud program. I will like to thank everybody that helped us and supported us. From my family, girlfriend, friends, other teams and so on. For the end, let me try to answer those 3 questions in post title.

Why me? Where me means you; you who is reading this post. If you have entrepreneur spirit, then Hekovnik and their programs are perfect for you. But aware! If you are doing this only because you’re bored or neighbor kids are also doing this and you also want to be cool, then don’t. I have saw many teams come and go, because they got stuck at first obstacle. Having a startup means being tough. Hekovnik will make you tough.

Why now? Look around. We live in wonderful, but strange world. Because we people like to complicate things, the number of problems is rising each day. Problem of not having a job, deadly diseases lurking upon us, not able to have a stable WI-FI, etc. See your startup as problem solver. Best problem solver. Hekovnik will help you accomplish that.

Why should I f*cking care? Few years ago I had no idea about Hekovnik. I didn’t even bother to take 30 mins away from my busy schedule to visit it. I feel bad about it. But when I finally visited it, I was surprised in a very positive way. With great people, wonderful programs & workshops and fast WI-FI you will achieve your dreams. The last one is most important. Joking aside, what I wanted to say is. Give it a change, Hekovnik has many wonderful things to offer, you just have to be willing to work and believe in your idea.