Mining for Meaningful Data
Every day businesses all over the world generate billions of log events. Sifting through this data to find key pieces of information is a vast, time-consuming task. Log management systems are typically expensive to buy and depend for their success on the technical skills of the data experts using them.
Poised to disrupt this is NovaUCD graduate company, Logentries. Its co-founders, Dr Trevor Parsons and Dr Villam Holub, have designed a user-friendly and accessible data access and analysis technology companies can buy on a cost effective SaaS basis.
Logentries' co-founders Dr Trevor Parsons and Dr Viliam Holub
“We provide a central location where all of an organisation’s log data can be easily accessed, managed and analysed,” explains Parsons, who is also the company’s chief scientist. “Important information can be dynamically tagged, automatically routed, and easily shared across teams and computing platforms. This model eliminates the need for an in-house data expert or specialist to interpret and use the data. Our service transforms difficult to manage data into a format that can be used to underpin a company’s strategic objectives while also giving deeper insights into key aspects of a business.”
Logentries is a spin-out from the Performance Engineering Laboratory in UCD’s School of Computer Science and Informatics. Today, the company’s service has tens of thousands of users in over 100 countries processing in excess of 10 billion log events a day. Customers include Fortune 100 companies, small and medium enterprises and individual practitioners.
“Our service lets companies focus on the data that matters most to them,” says Holub who is also the company’s chief technical officer. “In some cases less than 1% of log entries actually contain game-changing information. It usually lies hidden among the other 99%. Machine-generated log data is an important and fast-growing aspect of harnessing big data to help improve business and IT operations. However, rather than attempting to manage the ever-increasing amounts of logs and raw log data, Logentries focuses on finding and extracting the individual events that matter most. This allows anyone at any level to mine for valuable nuggets of information.”
Parsons and Holub met while working on an advanced academic research collaboration between UCD and IBM. “The aim of the specific collaboration with IBM in 2007-2009 was to take the knowledge of one of their raw data experts and package it in a way that their testers and developers could easily understand,” Parsons says. “We did this and subsequently rolled out the technology across numerous teams within IBM in Ireland, the US and China and the company quickly saw a big and beneficial return on that investment.”
Log data has traditionally been used primarily for security and compliance purposes but by the end of the project with IBM, Parsons and Holub could see other application opportunities from IT operations to web marketing. “Having built an applied solution for IBM, we set up Logentries in 2010 to build a new technology in the cloud,” Parsons says.
“We were the quintessential bootstrapping entrepreneurs with very little money and no salaries living on noodles,” he adds. “Having built up the knowledge working on the IBM project we were able to do the development work ourselves. We didn’t need to hire a big team unlike others trying to do the same thing in our space. In the end, we built the original version of our technology based on knowledge and experience rather than large amounts of capital. At the same time others spent millions of dollars developing technologies that were less feature rich and less reliable. We built our technology after 10 years of research in distributed systems in UCD and learned a huge amount working with companies like IBM. Just over a year ago we raised seed capital of $1 million and that allowed us to start bringing a technical team on board to accelerate product development.”
Logentries had initially received seed funding from Enterprise Ireland and has also been very successful in securing private funding. In October 2013 a funding round led by Polaris Partners raised $10 million. This will be used to accelerate product development, drive sales and support additional jobs. The company currently employs 20 people and is headquartered in Boston’s innovation district with offices in Dublin and Prague.
Holub and Parsons participated on the NovaUCD 2010 Campus Company Development Programme and Logentries won the NovaUCD 2010 Start-Up Company of the Year Award. In 2013 they received the NovaUCD Innovation Award in recognition of their global success and the impact Logentries has had in making log management and analytics accessible to virtually anyone.
“The aim of the NovaUCD Campus Company Development Programme was to help us commercialise the output of our research activities and develop it into a successful high-tech venture,” Holub says. “NovaUCD was very valuable in helping us to define and transform our ideas into a commercially feasible enterprise. Our connection with NovaUCD and UCD also gave us credibility. To be able to use the NovaUCD brand and say you’re a UCD spin-out company carries weight when you’re talking to investors and customers alike.”
Parsons says good timing has played a part in Logentries’ success to date. “The cloud has grown as we thought it might and log data has become as important as thought it would and unstructured data now makes up 90% of all data. At Logentries we are making that data accessible to everyone for a range of very valuable use cases, from developers to IT operations teams as well as to business users. We like to think that we did the PhDs so our users don’t have to.”