Alberto Saavedra 2.0

This is the phrase used by Eric Schmidt to explain Google’s portal-like moves when the new products to improve web search were announced and I am beginning to use it to explain my interest in search. I definitely don’t have a decade in search industry like Danny and I am not writing an afterward to a book that it’s the best reference to introduce search to people unfamiliar with the industry but I have some time following search engines technologies, in this post I will share my little journey with web search so far.

I was looking for a topic to write my engineering thesis back in 1998 and XML had just become a W3C recommendation so it got my attention. While looking where to apply it on the web I got frustrated with search engines so I figured I could do something with XML to help improve the search experience. I thought XML was going to change everything so I decided to build a search engine that indexes XML documents with a natural language interface.

The topic was relatively new and my allies in this journey were books like the XML Black Book, Natural Language Understanding, Managing Gigabytes to understand documents indexing and subscriptions to the Search Engine Watch and Sociedad Española para el Procesamiento del Lenguaje Natural to keep informed. After two years of dealing with the slow maturing of XML parsers I got a small set of news pages in Spanish which I manually marked up into NewsML format and build a thesaurus in XML format to get the synonyms of query terms.

The idea was simple, a user formulates a question which is analyzed and related terms are added to the query that is sent to the XML indexing engine, the results were ranked using term frequency. Every data structure was stored in XML so performance was slow but the experiments proof the concept that with XML markup and natural language query analysis relevant documents that were ignored became visible to the user.

In my thesis I concluded that the future search engine will be a meta-search with natural language capabilities that will query various vertical/specialized XML indexes and rank the results according to the question formulated and some link analysis using XLink, my assumption was that XML was going to be the dominant format in the web but it didn’t happen yet. With feed (RSS/Atom) search engines becoming more popular, meta search engines like gada.be getting more attention, XLink’s new Candidate Recommendation and a new NewsML 2 Architecture this scenario can still happen.

With Google delivering good enough results I was just a search power user until last year. Back to school and while looking for a topic for my MSc thesis I got interested in concept-based search and joined the Personal Digital Library Project in my school. My current thesis is about concept-based ranking using user contributed tags/labels and attention metadata in personal repositories.

And to stay up-to-date I listen to The Daily SearchCast and other WebmasterRadio shows, read the excellent coverage of the SES conferences available at the Search Engine Roundtable - read my own coverage of SEW Live Seattle in the next post - and subscribe to various search related blogs, if you don’t want to wait for Danny’s OPML this blogroll is a good starting point. Why search is so important now? After all, a lot of the information generated is searchable.

Labels: web-search

# posted by Beto : 1:42 AM  0 comments

The World Wide Web transformed the media industry; before its arrival, you had to use one of the traditional mass media services to get exposure but with the Web any individual can get a comparable audience with his/her website. The low-barrier to entry led to a proliferation of websites and in this scenario, the number of visits (traffic) became the success metric.

Advertising has been paying the bills for traditional media and banner advertising did the same for the web before the dot-com burst, it was the tool that businesses used to promote and brand their sites and companies like DoubleClick - through ad serving using the eyeballs model (CPM) - were mentioned by web entrepreneurs as the tactic to generate cash flow, something similar of what happens nowadays with Google AdSense.

In the web, advertisers have the metrics to evaluate their spending and banner ads were not being effective in generating traffic, so ad providers try to force exposure - following the tactics of traditional advertising model - using pop-ups; this move backfired because the most used web browsers incorporated pop-up blockers functionality in the application or via toolbars listening to users’ request who were being annoyed by the banner ads. This example of consumers rising power in the web is also impacting traditional media, advertisers are feeling that TV ads had become less effective and technologies like DVRs and iTV are beginning to impact the way TV advertising works.

What has shown effectiveness in the Web is the pay-per-click model introduced by Bill Gross, which is the success factor for the top public company in Computer Services industry. Google reported revenues of $2.23 billion generated by online advertising in the first quarter of 2006, with 41% of their total revenues coming from their AdSense partner network and 58% being generated by users who click ads in Google-owned sites. Yahoo, the second in market cap, reported revenues of $1.38 billion generated by their marketing services division in their financial results for first quarter.

The IAB reports that Internet Advertising revenues grow 30% in 2005 in the US and search spending has a 33.5% growth compared to 2004. With these healthy results from the leaders in paid search advertising, the projected increase in search marketing spending may fall short but they still do not show that the optimistic prediction made by Mark Kvamme in a keynote on Ad:Tech San Francisco this week - indicating that Internet Advertising will reach $35 billion in 2008 - is going to happen.

There is enough market for more players so the unusual spending projected by Microsoft for 2007 to support the software+services strategy and the upcoming announcements in next week’s MSN Summit related to their advertising program AdCenter combined with the Primetime exposure Ask is getting to gain attention could drive up even more online advertising market share numbers in US advertising, currently at 4.7 percent.

Web globalization is also validated with the figures reported, Google’s outside of the US revenues represent 42% of total and Yahoo’s international revenues are 30%; given that USA Internet users are only 18.3% of more than 1 Billion users worldwide and the growth of online advertising spending in Western Europe - with a 3.1% market share in media ad spending - and other regions, the future of the global online advertising industry looks promising, specially if you consider that local online advertising is just getting awareness.

Media economics were based on linear replication, but with current technologies easing the duplication of material there is no control on the generation of copies; while other business models get more mature the fact that Chris Dobson told Mark Evans in the Sympatico/MSN Digital Ad summit that online will be an ad world, not a subscription one is an indication that online advertising is recognized as the preferred monetization strategy in the Web nowadays.

Labels: online advertising

# posted by Beto : 1:34 AM  0 comments