Today we are launching Pository and Speclative, the first two toggle labs projects. toggle labs will be a whole new section on the site where we experiment with code, culture, open source, research, knitting, crafts and most importantly – ideas!
What are Pository and Speclative?
Pository decides how negative a page is and rates it accordingly. If the page fails then Pository recommends you read something a little happier. Speclative works in a similar way but looks for speculative language instead. Perhaps the title of this post makes sense now?
Where did the idea come from?
Last year we spent a great deal of time commuting. On a dreary three hour commute your eyes get drawn to the newspaper headlines on the many free newspapers. If you have ever been on a train at rush hour you will appreciate how easy it is to get sucked into articles being read by the person opposite. We were commuting whilst swine flu was a hot topic and the headlines read: Swine flu ‘could kill up to 120m’ and similar. It was clear to see that most of the articles were based on speculation and for the most part were written to spread misery and negativity. Could we do something about it?
A few months into our commute we started to collect the newspapers discarded on the trains at the end of each day and look for patterns in language. Whilst we could not filter the stories in the real world we decided to take our battle to online where it is easier to analyse the data on a page. The idea was born.
How do they work?
As soon as you click on one of the bookmarkets the URL of the current page your viewing is sent to our web server. In a fraction of a second we retrieve the web page as plain text (this only works on publicly accessible pages). For Pository we have complied a list of positive (happy) and negative (sad) words and weighted them accordingly. We then count the occurrences of those words and from that we can score the page. Speclative works in a similar way except the keyword lists are different — based on fact and speculation. Once we have a score we can make a recommendation as to how speculative or negative the article is.
The microsites are built using HTML5 and on the server side we are using PHP5 with the excellent PHP Simple HTML DOM Parser library. The bookmarklets make use of jQuery and will auto update once you have them installed. The bookmarklets have been tested to work on all modern browsers except for Internet Explorer (support for IE will be available soon).
What comes next?
For now we are very interested in both the data we collect and the feedback we receive. Over time we hope to improve the scoring and weighting to reduce the number of “unknown” results.
The data collected from the two bookmarklets will allow us to make generalisations about various news sources. Which websites provide happier news articles? Which websites write articles based on fact? Which websites write articles based on speculation? Once we have enough users of the service we will be able to answer these questions in real-time and eventually provide recommendations of other sites to visit: “Is the current page you’re reading too sad? Try this website that has written about a similar article in a positive way.”
Once the databases have filled up we hope to provide interactive, real-time visualisations of the results. These will inform our research into Internet filtering. As the Internet swells with content the ability to quickly determine the usefulness of an article will be very important.
Speclative and Pository were made to be good fun to use and they were certainly enjoyable to build. More details of how to install, use and provide feedback on the bookmarklets can be found at postitory.scott.ee and speclative.scott.ee.
Update 10/02/2013: The code for these projects is now available on GitHub.