A startup in New York City called Dataminr is using Twitter to potentially save lives by detecting news faster through analysis of big data. The company has been the first to generate news alerts on a number of events that had the potential to save lives including tornadoes, explosions, forest fires and various shootings. In March, when a gas explosion destroyed a large section of a Harlem block in New York City, Dataminr verified the event had occurred within 200 seconds. It took major news stations over 20 minutes to get hold of the breaking story. In April it helped Boston authorities in a strategy of live-tweet tracking to avoid another attack on the city’s annual marathon.
Back in January CNN announced a partnership with Dataminr to “alert journalists to information that’s emerging on Twitter in real time,” according to Dataminr CEO Ted Bailey. For Dataminr, the social media company’s data set is perfect for spotting trends early. CNN has already used the information from the New York startup on numerous occasions such as a Maryland mall shooting, a United Airlines Beijing-bound flight from New Jersey that had to be called back for turbulence and a Brooklyn cupcake shop that was harassed with calls about Obamacare.
The general manager of CNN Digital explained that CNN has no financial interest in Dataminr and no commercial interest in the product. In other words, the company came to an arrangement for exclusive, early access to the product, but when it launches fully later this year, CNN will just become another customer, although they will be a customer with a definitive early-mover advantage heading into the future.
As a strategic partner of Twitter, Dataminr has access directly into the company’s firehouse to find relevant information that it can share with its partners in finance, news and the public sector. Now the question is how they distinguish what is vital information from all the noise flying about on the internet.
Dataminr used its big data connections and created an algorithm that saves time by understanding the interrelationships in the everyday lives of Twitter’s users. “People are sensors out there and if you look at our archive over time there’s a flow and pattern for every story there has ever been,” said CEO Ted Bailey. There are over 500,000,000 million tweets a day and Dataminr has access to all of them. They take those tweets and then score, tag and classify them through a number of categories including user reputation, user interests, topic density, user influence and spam quotient, among others. Then that information is clustered together by even more factors such as location, momentum and the novelty of the information as well as the co-occurrences of information.
All of this is then evaluated and detected by a filter that looks at real-time situational awareness and historical comparisons simultaneously. That information is then made into an alert that can vary in scale from emerging trends, market-moving information and breaking news which are further evaluated to have an urgency level determination of low to high. Those alerts and their level of urgency are analyzed again by topics, sectors, geographies, languages and more, which ultimately sends them to clients through user interface notifications, emails, instant messages or directly to client server systems. It is a very complex chain of events that is happening at amazing speeds.
When someone says “information on Twitter can be unreliable” they are overlooking an exclusive attribute that the company has which is the ability to verify itself. When looking at one tweet from an unverified source it can be difficult to be convinced about its reliability. But when tweets are evaluated as an aggregate and then clustered together the adjacent tweets can be combined by location to present a very convincing data-description of a breaking news story.
Dataminr is just one of an increasing number of companies who are built on analyzing data from Twitter. The difference is that most others do not have their primary emphasis on real time news updates. An executive from the social media company said “Dataminr’s technology is very advanced; every day there is another example of how far ahead they are.”
Big data is changing the way we live and through Twitter Dataminr has created a platform to save people money and in extreme situations their lives. Both companies want people to understand how social media can affect lives instantly and therefore want more people to tweet about what they are seeing around them.
By B. Taylor Rash