QCRI and Al Jazeera launch predictive web analytics platform for news

News organisations have vast archives of information, as well as a number of web analytic tools that aid in allocating editorial resources to cover different news events,andcapitalise on this information. These tools allow editors and media managers to react to shifts in their audience’s interest, but what is lacking is a tool to help predict such shifts.

Qatar Computing Research Institute (QCRI) and Al Jazeera are announcing the launch of FAST (Forecast and Analytics of Social Media and Traffic), a platform that analysesin real-time the lifecycle of news stories on the web and social media, and provides predictive analytics that gauge audience interest.

“The explosion of big data in the media domain has provided QCRI an excellent research opportunity to develop an innovativeway to derive value from the information,” said Dr Ahmed Elmagarmid, Executive Director of QCRI. “Together with our valued partner, Al Jazeera, the QCRI team has developed a platform that will help shift the way media does business.”

“Al Jazeera English’s website thrives on good original content in news and features, dynamic ways of creativity through interactive and crowd sourcing methods, and up-to-date social media tools. We welcome working with QCRI in developing FAST as it allows us to understand the consumption of news and what is expected to do well in driving traffic forward. Analytics in predicting the future trend of a web story is a crucial component in understanding web traffic, this initiative is a component we welcome,” said Imad Musa, Head of Online for Al Jazeera English.

The study of consumption patterns of online news has attracted considerable attention from the research community for more than a decade, primarily making predictions on patterns as single time series to determine website traffic, number of visits, number of comments, and personalised news recommendations among others. Predicting user behaviour around news articles is valuable for a news organisation as it allows them to deliver more relevant and engaging content, as well as improve the allocation of resources to developing stories.

FAST introduces a unique approachto prediction by integrating different user interactions to a news article, including website visits, social media reactions, and search and referrals in order to forecast the number of page viewsan article will receive during its effective lifetime, which is approximately three days for most articles. This hybrid observation method is based on qualitative and quantitative analysis that determines typical patterns in the lifecycle of news.

The underlying algorithms, which are the result of joint research by scientists at QCRI, Al Jazeera, Carnegie Mellon University and theMIT Center for Civic Media, have been validated using vast amounts of data made available by Al Jazeera English. The platform accurately models the overall traffic an article will receive by observing the first thirty to sixty minutes of social media reactions. Achieving the same prediction accuracy by using data fromvisits alone would require at least three hours of data.FAST continuously learns to produce more accurate predictions as data from the most recent related articles streams into the system.

“One of the main conclusions from our research is that social media reactions cannot be ignored when producing traffic predictions,” said Dr Carlos Castillo, Senior Scientist in QCRI’s Social Computing team.  “You need to take into account not only the number of Facebook shares and tweets each article receives, but also the richness of the discussion around an article in Twitter. This leads to much more accurate predictions than simply extrapolating from current pageviews.”