Innovation, community and and technology experts from Yahoo! and the BBC came together on 27th April 2006 to explain their own efforts in the development of web services and mash ups and discuss the opportunities and challenges in this area...

Report by Deirdre Molloy

[Register and post your own comments on this report below...]

Introducing the event, chair Greg Tallent of, lecturer in eBusiness at London South Bank University cited http://www.emilychang.com/go/ehub, which lists mash ups such as Google Maps + Flickr alongside many others.

Simon Willison � Yahoo! Technology Development Group

As Simon framed it, the Yahoo Technology group are more concerned with looking a bit ahead of the curve, and they have started developing a �web of data�, moving beyond the current �web of pages�. This is all about science and applications that, instead of just looking up data, allow it to be used in new and unexpected ways.

Of the recent Yahoo! acquisitions � Flickr, Delicious, Upcoming � a notable shared characteristic is that they all have APIs (application programmable interfaces) and Yahoo are also trying to open up APIs for Yahoo�s other services and products as well, for example its data copy of the entire web.

They�ve created incredibly powerful applications � Yahoo Search, Yahoo Finance and Yahoo Maps, and the aim is to expose these huge amounts of data and enhance their value, although currently Yahoo�s API content is currently only for non-commercial use. Flickr is making money out of APIs via the ability to order photos and professionally bound books of the photos (albeit delivered by external companies). But Yahoo! is looking to make their API appliances more widespread and clearer for businesses and entrepreneurs, so you�ll be able to build upon their services in the future. You can start experimenting now at developer.yahoo.net and they�re presently scoping out the business models for the future.

A business case for APIs?

Yahoo�s Photo site is much bigger then Flickr. Releasing an API that allows you to access Yahoo photos on behalf of Yahoo users requires the ability to authenticate your services against Yahoo�s existence. From a hacker point of view this is fantastic and from a business point of view it will be important moving into the future, as will having a gallery of applications.

When will the commercial case for these applications become clearer, asked Zaeem Maqsood of First Capitall. Simon said they dealt with such enquiries on a case-by-case basis and recommended contacting Yahoo! Is there a limit to the extent of opening up before it stops becoming competitive, another delegate asked. Amazon have been releasing commercial APIs for a while and Flickr has benefited immensely from open APIs, Simon replied, citing the Linux-Flickr Uploader, while for other niche platforms, he reflected, it perhaps isn�t yet economically viable to do internally.

Emmanuel Ide of Jazar wondered if there will be a standardization of APIs and Simon observed that the most common standards are Atom and RSS feed APIs. Ning was cited, as was a start up called www.ookles.com, a photo-sharing service that cloned the Flickr API in its entirety.

Is the "Web of data" the semantic web?

The issue was raised as to whether the web of data was the semantic web, and do Yahoo! as a company know how they will structure that data? The web will look the same but there will be another level of data that developers can access, Simon said. What of metadata, standards and the Dublin Core? Simon reckoned that the W3C revolves around RDF and it just hasn�t taken off in a big way. The race is on to discover newer, simpler ways of making data extractable by machines, and Microformats is one to watch in this field and gaining a lot of buzz because it�s simple for people to implement. At the same time it�s important to be aware of the consequences of making that data available (eg. switching off automatic RSS feeds).

James Cooper noted that with MySpace you can plug into it by just using a little widget that has the same effect as an API � so what�s the difference? Simon said that MySpace has taught us a great amount about what people want to do with their sites (flashing backgrounds and glittering text!). They have no open APIs but a universal API that lets you paste code into sites. It would be fascinating if they opened up their APIs to let others do the stuff they want to.

Tom Loosemore � Project Director, BBC 2.0

Tom spoke of the �vicar in trainers� approach to innovation taken by the BBC so far, but noted that he�s just been granted his new job title, part of a wider move beyond being a broadcaster to a provider of media. He recalled that when his personal non-BBC project TheyWorkForYou.com won an award his boss asked him why they were not doing that here. They need to be able to innovate at the edge if they are to secure their future and their relevance, Tom observed.

The semantic web will reveal the real power of the web, and it you want to thrive in a web of data you need to be part of it, Tom stressed. His team spent a lot of effort over 6 months hunting down data � they did some work and looked about but there didn�t seem to be much at hand. When they looked around to see what others were doing, they found a guy who had scraped BBC content and they linked to it. Instantly Wikipedia and the blogosphere were talking about the story. And it did of course break their copyright.

In May 2005 they launched the BBC Backstage project � a very vicar in trainers site, with a blog attached. It was more of a statement of intent than a full-blown proposition. But people have done some amazing stuff with it, Tom remarked, for example, the �Was This (weather) Forecast Right or Not?� mash-up.

It's amazing what librarians can spark off...

Then about 6 months ago they stumbled into a BBC librarian and he showed them how they catalogue their data. They�ve been categorising the BBC archive for the last 70 years and had religiously kept updating it. That resulted in the launch yesterday (26thth April) of the BBC Programme Catalogue. It looks ugly but the data is far-reaching. Tom even discovered through it that his father had been on TV in 1988 but he�d never told Tom about it.

So how far do you let it go? Tom said he�d like all the data to be available because then you are in a very strong position in the value chain if your data or API becomes the standard.

Rob McKinnon mentioned he was working the New Zealand version of TheyWorkForYou.com, in terms of the common idea of remixing content to create new stuff. He raised the open source model of collaborative creation of value. If you take such community-based peer production models enabled by internet plus broadband, does this amount to a new economic production model that people can benefit from? Tom concurred that the web is transformational in terms of social and economic value creation. The model also applies to public value, he continued. With just a few volunteers you could do the same thing.

It�s not so much about who builds the API, Simon agreed, but about what you can do with it, eg the Dingy or rowing society � all their info is locked up in PDF documents � and this unlocking of usefulness and value scales for all different sizes of diverse organizations.

Discussion & Q&A with the audience

What are the responsibilities of the companies creating this web of data Tom Coates mused. At the launch of BBC Online, Simon noted, they made the great decision to give every article an unbreakable link. But the delegate from Guardian Unlimited was worried that his boss would ask him how the data will be protected and what else can we do with it? Simon responded that it�s worth making the investment in structuring your archive data, but Tom Coates added that on the IP side you just can�t control it, people will find a way, but you do have legal protection and various licences are available.

What about the category of mash ups that are about mashing services and not just data (eg. content management services)? Simon flagged-up Ning, a site that makes it easier for you to create (or clone) APIs. But the really interesting thing, he reckoned, is that from the technological point of view it�s just one enormous mesh.

Are mash-ups more than toys?

Zaeem Maqsood commented that Amazon released their API a few years ago but it hasn�t rocked our world, and there�s been a couple of cool mobile apps. So what are we talking about here � are we just creating toys, he asked. The Amazon click-through from other sites pays the sites that deliver these links, Simon countered. Take Delicious Monster - their Delicious Library software product lets you scan barcodes on your books/DVDs/CDs using a iSight camera and then downloads more information about them from Amazon's API. It also recommends other books you might like, and lets you buy them from Amazon. A great example of commercial usage of an API, in Simon's opinion. Tom Loosemore noted that we�re also seeing large companies who have developed good 1.0 models now tinkering with news-based APIs.

For Tom, SXIP Identity were the most interesting company in this space, as they are focused on federated identity. Just as Google owns the search query level of the internet, whoever can own those other layers of the web apart from front-end websites will make a lot of money. Microsoft Passport is also built around the notion that people are getting fed-up with ever-multiplying log-ins.

The BBC, however, is about delivering public value at its core, and making media, and there is no route currently for people to exploit APIs commercially. The obligation is to provide value for money, so they did not buy rights in their entirety, which made sense in the traditional model of broadcasting, he noted.

Tom Coates observed that if the commercial arm of the BBC controls the reselling of content, to what extent can we expect BBC Worldwide to be part of the 2.0 shift? Not any time soon was Tom Loosemore�s estimation.

Will mash-ups go mainstream?

Thayer Driver of Chinwag Jobs asked the speakers for their personal favourite mash ups and what they saw happening the coming year ahead. Tom said that when developing the BBC Programme Catalogue, a developer shared early developments with his mates and one of them thought they could create graphs of politicians� careers; there was also a greasemonkey script produced that plots the careers of different celebrities concurrently. To the question of whether it will ever go mainstream, Tom reckoned they could do a lot of great, popular stuff with the government data.

Simon commented that the most useful mash up was the Craigslist & Google Maps apartment mash up HousingMaps.com, a really useful combination of data. Andy Budd of Clearleft noted that ononemap.com does the same thing with estate agents in the UK. Rob McKinnon reasoned that if you own the domain space or layer you win. He spoke of the New Zealand company Zoomin who have added a Flickr clone to Google Maps NZ whereby the street name of every street in New Zealand has its own Zoomin page and that page is now the top result of every search for a street name on Google NZ. If your business is on one of those streets maybe you should think about contacting Zoomin about some advertising, he added.

Sirish Reddi asked when will all this matter to people who don�t have mobile phones or the internet? Greg Tallent added that Web 2.0 is also about social connections. The real value is the Friendster or MySpace way � it�s about creating or sharing your identity. There have also been interesting proposals for the Olympics � connecting people through sport. But Greg reckoned that all platforms and channels will persist.

Where next for the open web?

Tom reflected that what we�re talking about is how do you innovate and what do you need to enable this to happen? Standards was a suggestion from a delegate but Tom disagreed � what we need is openness, he said.

The UK is the biggest user (per head of the population) of BitTorrent in the world, Simon remarked, and people�s interaction won�t change. Tom countered this, saying people�s attitudes and behaviours are changing because of increasing choice and opportunity [a remark echoed by Adriana Cronin Lukas two months later at Content 2.0 - Ed]. Standards cannot be fixed when the environment is not mature, he continued, and Tim Berners-Lee would now find it hard to argue that the web has not grown as a mess. Standards emerge and imposing them stifles innovation.

Tom Coates said that the appearances of API�s is the first sign of the movement forward towards structured data because you need it in order to repurpose the content, and then the value of API�s becomes clearer. Emmanuel Ide said that the BBC even has different XML standards internally and Tom Loosemore confirmed this, adding that the interesting question is when do people start sharing the same standards? He hoped that people will start cloning their RDF standards. Simon noted that the development of web data came in three broad stages: unstructured, structured and standardised, and he stressed that we're still in the phase of trying to get it structured.

See the original EVENT PAGE

About the Speakers:

Simon Willison - Technology Development, Yahoo!
Simon Willison works far Yahoo! on the Technology Development team. He is an experienced client- and server-side developer and maintains a long running technical weblog. He is one of the hackers behind Django, the open-source Python web framework aimed at "web developers on journalism deadlines". Simon will talk about the web services being developed by Yahoo! and the opportunities for companies and developers to work with Yahoo! content.

Tom Loosemore - Project Director, BBC 2.0
Tom Loosemore is a senior manager at BBC New Media, where he has championed the building of solid foundations underpinning bbc.co.uk, including search, single sign-on and content management. Tom is one of the founders of a small cabal of volunteer 'civic hackers' dedicated to developing sites (such as FaxYourMP.com and TheyWorkForYou.com) that poke British democracy with an internet-shaped stick in the hope that one day it might wake up. Tom will be speaking about the BBC Backstage project and community building.

Chair: Greg Tallent - Senior Lecturer, E-business, LSBU and Bearstorm
Greg is Senior Lecturer in E-business at London South Bank University. His main research interests are in the web phenomenon of 'social networking', in particular how the Internet has enabled identity and personality to be shared by people within interest groups. He has lectured extensively on: Media Disruption in the Internet Age; What is Web 2.O?; and A Philosophical Framework for Web Behaviour.

---------------

Subsrcibe to the Beers & Innovation blog

Overview of Beers & Innovation 1: UK Start-Up Culture
Overview of Beers & Inovation 2: User Generated Content
Overview of Beers & Innovation 3: Web Services & Mash-Ups
Overview of Beers & Innovation 4: RSS Frontiers

Read the Beers & Innovation 1: UK Start-Up Culture report
Read the Beers & Innovation 2: User-Generated Content report

To be kept posted on all future B&I nights, sign up for the fortnightly NMK Newsletter (just drop your email address into the third box down on the right hand side of the homepage).

About Beers & Innovation:
This is an ongoing series NMK are producing, with each Beers & Innovation focusing on a particular key issue for / sector of the UK's innovation and technology scene. The next one will be announced soon. Regular updates and relevant discussions can be found on the blog. For enquries about this or future B&I nights, email deirdre.molloy (AT) nmk.co.uk - we welcome all your comments, ideas and feedback!