Alberto Saavedra 2.0

Everyone involved in the SEO industry wants to know what factors are taken into consideration when search engines rank pages, we are not going to be able to know all the factors or how they are combined because that is the secret sauce of search engines but that does not stop us to try to figure out which ones are the most relevant.

We have some hints about the components considered by Google thanks to their related patent applications: Information Retrieval based on Historical Data, Ranking Blog Documents, Ranking Businesses at a Location in local searches, Personalization and agents that analyze content aggregation sources’ reputation; but these are too many factors to explain to our customers who just need to get better positioning on search engines.

So when the SEO’s A-list share their opinions on the most-known ranking factors you have a must-read article: Search Engine Ranking Factors V2 is the document that represents the collective wisdom of 34 SEO leaders, it is digg-worthy.

Danny Sullivan takes a look at the level of difficulty to control the most important factors ranked by the experts, really useful to introduce SEO to beginners. The factors ranked by importance and categorized as presented in the document are the following:

Keyword Use Factors

1. Keyword use in title tag
2. Keyword use in the body text
3. Relationship of body text content to keywords
4. Keyword use in H1 tag
5. Keyword use in domain name
6. Keyword use in URL
7. Keyword use in H2, H3, H(x) tags
8. Keyword use in alt tags and image titles
9. Keyword use in bold/strong tags
10. Keyword use in meta description tag
11. Keyword use in meta keywords tag

Page Attributes

1. Link popularity within the site’s internal link structure
2. Quality/Relevance of links to external Sites/Pages
3. Age of document
4. Amount of indexable text content
5. Quality of the document content
6. Organization/Hierarchy of document flow
7. Frequency of updates to page
8. Number of trailing slashes (/) in URL
9. Accuracy of spelling & grammar
10. HTML validation of document (to W3C Standards)

Site/Domain Attributes

1. Global link popularity of Site
2. Age of Site
3. Topical relevance of inbound links to Site
4. Link popularity of Site in topical community
5. Rate of new inbound links to Site
6. Relevance of Site’s primary subject matter to query
7. Historical performance of Site as measured by time spent on page, clickthroughs from SERPs, direct visits, bookmarks, etc.
8. Manual Authority/Weight given to Site by Google
9. Top Level Domain extension of Site
10. Rate of new pages added to Site
11. Number of queries for Site/Domain over time
12. Verification of Site with Google Webmaster Central

Inbound Link Attribute

1. Anchor text of inbound link
2. Global link popularity of linking Site
3. Topical relationship of linking Page
4. Link popularity of Site in topical Community
5. Topical relationship of linking Site
6. Age of link
7. Text surrounding the link
8. Internal link popularity of linking page within Host Site/Domain
9. PageRank (as measured by the Google Toolbar) of linking page
10. Domain Extension of linking Site
11. Temporal link attributes

Negative Crawling/Ranking Attributes

1. Server is Often Inaccessible to Bots
2. Content Very Similar or Duplicate of Existing Content in the Index
3. External Links to Low Quality/Spam Sites
4. Participation in Link Schemes or Actively Selling Links
5. Duplicate Title/Meta Tags on Many Pages
6. Overuse of Targeted Keywords
7. Very Slow Server Response Times
8. Inbound Links from Spam Sites
9. Low Levels of Visitors to the Site

Labels: web-search

# posted by Beto : 3:06 AM  0 comments

This is the phrase used by Eric Schmidt to explain Google’s portal-like moves when the new products to improve web search were announced and I am beginning to use it to explain my interest in search. I definitely don’t have a decade in search industry like Danny and I am not writing an afterward to a book that it’s the best reference to introduce search to people unfamiliar with the industry but I have some time following search engines technologies, in this post I will share my little journey with web search so far.

I was looking for a topic to write my engineering thesis back in 1998 and XML had just become a W3C recommendation so it got my attention. While looking where to apply it on the web I got frustrated with search engines so I figured I could do something with XML to help improve the search experience. I thought XML was going to change everything so I decided to build a search engine that indexes XML documents with a natural language interface.

The topic was relatively new and my allies in this journey were books like the XML Black Book, Natural Language Understanding, Managing Gigabytes to understand documents indexing and subscriptions to the Search Engine Watch and Sociedad Española para el Procesamiento del Lenguaje Natural to keep informed. After two years of dealing with the slow maturing of XML parsers I got a small set of news pages in Spanish which I manually marked up into NewsML format and build a thesaurus in XML format to get the synonyms of query terms.

The idea was simple, a user formulates a question which is analyzed and related terms are added to the query that is sent to the XML indexing engine, the results were ranked using term frequency. Every data structure was stored in XML so performance was slow but the experiments proof the concept that with XML markup and natural language query analysis relevant documents that were ignored became visible to the user.

In my thesis I concluded that the future search engine will be a meta-search with natural language capabilities that will query various vertical/specialized XML indexes and rank the results according to the question formulated and some link analysis using XLink, my assumption was that XML was going to be the dominant format in the web but it didn’t happen yet. With feed (RSS/Atom) search engines becoming more popular, meta search engines like gada.be getting more attention, XLink’s new Candidate Recommendation and a new NewsML 2 Architecture this scenario can still happen.

With Google delivering good enough results I was just a search power user until last year. Back to school and while looking for a topic for my MSc thesis I got interested in concept-based search and joined the Personal Digital Library Project in my school. My current thesis is about concept-based ranking using user contributed tags/labels and attention metadata in personal repositories.

And to stay up-to-date I listen to The Daily SearchCast and other WebmasterRadio shows, read the excellent coverage of the SES conferences available at the Search Engine Roundtable - read my own coverage of SEW Live Seattle in the next post - and subscribe to various search related blogs, if you don’t want to wait for Danny’s OPML this blogroll is a good starting point. Why search is so important now? After all, a lot of the information generated is searchable.

Labels: web-search

# posted by Beto : 1:42 AM  0 comments

When new technologies arrive, metaphors are used to explain new behavior and functionality they bring by associating technical jargon with concepts already known. Internet and PCs are the pillars of the Information Age but it was only with the World Wide Web and links as the primary user interface metaphor that a user realized there is a window - the browser - to access information in the rest of the world. Nowadays, the knowledge society concept is not generally understood but thanks to web search - as the user interface mechanism - it is becoming clear; the results of a query provide relevant resources to any given interest which is how knowledge societies get together.

The first step to knowledge building is an adequate organization of information. 10 years ago, Netcraft Web Server Survey reported more than 150K sites active and there were two approaches used to organize the amount of information available: directories, a human-powered effort trying to classify web sites and search engines, which were gathering the text available in web pages at reach. The first approach could not handle the exponential growth of the web and the second one was in danger because the ranking mechanism was being tricked by term repetition when a popularity approach came to rescue.

Back in 1998, links were generated to lead the reader to a resource that was worth to check out if more information was needed. This human-powered action is a vote for the linked site and algorithms like HITS by Jon Kleinberg and PageRank by Larry Page and Sergey Brin use this concept to provide a better ranking of the results, this approach is one of the factors that made Google the preferred search engine.

In The Search, John Battelle explains how Google and its rivals had been transforming our culture. When you need some information about any topic it is almost a reaction to open your favorite search engine in the browser and ask for resources about that interest, sometimes you may not find exactly what you are looking for but it is a given that you will find something related at least. According to PEW search engine use is the second Internet activity on a typical day and it is probably the top activity performed in the browser.

This usage popularity makes commercial sites wish to get better visibility on search engines, to cover this need a industry called search engine marketing has been established which focuses in two areas: paid search or search advertising and search engine optimization. SEO practitioners analyze how search engines work and suggest strategies to get better ranking positions on determined queries, sometimes this knowledge is used to deliberately affect results so search engines are constantly adjusting algorithms to avoid these attempts.

Since PageRank is query independent it was implemented faster than other link-based ranking algorithms but it can also be compromised by link spamming - automatic generation of votes. Google as the industry leader has been using other refinements to maintain relevancy like the Hilltop algorithm to enhance result accuracy using a subset index to identify expert documents relevant to the query; historical data from web sites/pages to determine trust and freshness; search user behavior analysis to include personalized factors and tools for publishers like sitemaps to coordinate how pages are included in Google index.

The search box has become a user interface metaphor in web sites, and is moving from HTML code to the browser toolbar and ultimately to the desktop, making it the de-facto mechanism used to ask for information from computers. Although search engines provide a set of operators to allow query refinement, an average search user is not aware of them - only 2.7 words are usually present in a query - and this is the current search engines’ challenge: deliver the highest user satisfaction when users provide so little information about it. There are some UI experiments in place trying conversational approaches - related searches suggestion - and contextual - information supply - that are going to influence the users expectations when using the next generation of web search applications.

Web Search 1.0 established the way we request knowledge resources from Internet using common knowledge, my take is that the next innovation flow will focus on specific knowledge by improving relevancy in vertical markets, communities and private repositories - enterprise or personal - and you will read more about this on upcoming posts, in the mean time if you want to keep an eye on search engine industry news, the Search Engine Watch Blog is the place to visit.

Labels: web-search

# posted by Beto : 1:28 AM  0 comments