Starting today Google Webmaster Tools helps you better control the country association of your content on a per-domain, per-subdomain, or per-directory level. The information you give us will help us determine how your site appears in our country-specific search results, and also improves our search results for geographic queries.

We currently only allow you to associate your site with a single country and location. If your site is relevant to an even more specific area, such as a particular state or region, feel free to tell us that. Or let us know if your site isn’t relevant to any particular geographic location at all. If no information is entered in Webmaster Tools, we’ll continue to make geographic associations largely based on the top-level domain (e.g. .co.uk or .ca) and the IP of the webserver from which the context was served.

For example, if we wanted to associate www.google.com with Hungary:

But you don’t want www.google.com/webmasters/tools” associated with any country…

This feature is restricted for sites with a country code top level domain, as we’ll always associate that site with the country domain. (For example, google.ru will always be the version of Google associated with Russia.)

Note that in the same way that Google may show your business address if you register your brick-and-mortar business with the Google Local Business Center, we may show the information that you give us publicly.

This feature was largely initiated by your feedback, so thanks for the great suggestion. Google is always committed towards helping more sites and users get better and more relevant results. This is a new step as we continue to think about how to improve searches around the world.

We encourage you to tell us what you think in the Webmaster Tools section of our discussion group.

Great post Posted by Mickey Kataria, Google Zürich

“Since the launch of Sitemaps, webmasters have been asking if they could submit their Sitemaps for multiple hosts on a single dedicated host. A fair question — and now you can!

Why would someone want to do this? Let’s say that you own www.example.com and mysite.google.com and you have Sitemaps for both hosts, e.g. sitemap-example.xml and sitemap-mysite.xml. Until today, you would have to store each Sitemap on its respective host. If you tried to place sitemap-mysite.xml on www.example.com, you would get an error because, for security reasons, a Sitemap on www.example.com can only contains URLs from www.example.com. So how do we solve this? Well, if you can “prove” that you own or control both of these hosts, then either one can host a Sitemap containing URLs for the other. Just follow the normal verification process in Google Webmaster Tools and any verified site in your account will be able to host Sitemaps for any other verified site in the same account.”

Read this post here!

Sitelinks are extra links that appear below some search results in Google. They serve as shortcuts to help users quickly navigate to the important pages on your site.

Selecting pages to appear as sitelinks is a completely automated process. Our algorithms parse the structure and content of websites and identify pages that provide fast navigation and relevant information for the user’s query. Since our algorithms consider several factors to generate sitelinks, not all websites have them.

Now, Webmaster Tools lets you view potential sitelinks for your site and block the ones you don’t want to appear in Google search results. Because sitelinks are extremely useful in helping users navigate your site, we don’t typically recommend blocking them. However, occasionally you might want to exclude a page from your sitelinks, for example: a page that has become outdated or unavailable, or a page that contains information you don’t want emphasized to users. Once you block a page, it won’t appear as a sitelink for 90 days unless you choose to unblock it sooner. It may take a week or so to remove a page from your sitelinks, but we are working on making this process faster.

To view and manage your sitelinks, go to the Webmaster Tools Dashboard and click the site you want. In the left menu click Links, then click Sitelinks.

Thanks for your feedback and stay tuned for more updates!

Common feedback we hear from webmasters is that you want us to improve the freshness of the data in Webmaster Tools. Understood. We’ve increased the update frequency for your verified sites’ data, such as crawl, index, and search query stats. Much of this data depends on the content of your site. If your content doesn’t change very often, or if you’re not getting new links to your site, you may not see updates to your data every time you sign in to Webmaster Tools.

Please continue to post your Suggestions & feature requests in the Webmaster Help Group. It’s one of our most important sources of feedback from the webmaster community. We seriously take it seriously.

The quality of your snippet — the short text preview we display for each web result — can have a direct impact on the chances of your site being clicked (i.e. the amount of traffic Google sends your way). We use a number of strategies for selecting snippets, and you can control one of them by writing an informative meta description for each URL.

Why does Google care about meta descriptions?
We want snippets to accurately represent the web result. We frequently prefer to display meta descriptions of pages (when available) because it gives users a clear idea of the URL’s content. This directs them to good results faster and reduces the click-and-backtrack behavior that frustrates visitors and inflates web traffic metrics. Keep in mind that meta descriptions comprised of long strings of keywords don’t achieve this goal and are less likely to be displayed in place of a regular, non-meta description, snippet. And it’s worth noting that while accurate meta descriptions can improve clickthrough, they won’t affect your ranking within search results.

Snippet showing quality meta description

Snippet showing lower-quality meta description



What are some good meta description strategies?
Differentiate the descriptions for different pages
Using identical or similar descriptions on every page of a site isn’t very helpful when individual pages appear in the web results. In these cases we’re less likely to display the boilerplate text. Create descriptions that accurately describe each specific page. Use site-level descriptions on the main home page or other aggregation pages, and consider using page-level descriptions everywhere else. You should obviously prioritize parts of your site if you don’t have time to create a description for every single page; at the very least, create a description for the critical URLs like your homepage and popular pages.

Include clearly tagged facts in the description
The meta description doesn’t just have to be in sentence format; it’s also a great place to include structured data about the page. For example, news or blog postings can list the author, date of publication, or byline information. This can give potential visitors very relevant information that might not be displayed in the snippet otherwise. Similarly, product pages might have the key bits of information — price, age, manufacturer — scattered throughout a page, making it unlikely that a snippet will capture all of this information. Meta descriptions can bring all this data together. For example, consider the following meta description for the 7th Harry Potter Book, taken from a major product aggregator.

Not as desirable:

There are a number of reasons this meta description wouldn’t work well as a snippet on our search results page:

• The title of the book is complete duplication of information already in the page title.
• Information within the description itself is duplicated (J. K. Rowling, Mary GrandPré are each listed twice).
• None of the information in the description is clearly identified; who is Mary GrandPré?
• The missing spacing and overuse of colons makes the description hard to read.

All of this means that the average person viewing a Google results page — who might spend under a second scanning any given snippet — is likely to skip this result. As an alternative, consider the meta description below.

Much nicer:

What’s changed? No duplication, more information, and everything is clearly tagged and separated. No real additional work is required to generate something of this quality: the price and length are the only new data, and they are already displayed on the site.

Programmatically generate descriptions
For some sites, like news media sources, generating an accurate and unique description for each page is easy: since each article is hand-written, it takes minimal effort to also add a one-sentence description. For larger database-driven sites, like product aggregators, hand-written descriptions are more difficult. In the latter case, though, programmatic generation of the descriptions can be appropriate and is encouraged — just make sure that your descriptions are not “spammy.” Good descriptions are human-readable and diverse, as we talked about in the first point above. The page-specific data we mentioned in the second point is a good candidate for programmatic generation.

Use quality descriptions
Finally, make sure your descriptions are… descriptive. It’s easy to become lax on the quality of the meta descriptions, since they’re not directly visible in the UI for your site’s visitors. But meta descriptions might be displayed in Google search results — if the description is high enough quality. A little extra work on your meta descriptions can go a long way towards showing a relevant snippet in search results. That’s likely to improve the quality and quantity of your user traffic.

In recent months, there’s been a noticeable increase in the number of compromised websites around the web. One explanation is that people are resorting to hacking sites in order to distribute malware or attempt to spam search results. Regardless of the reason, it’s a great time for all of us to review helpful webmaster security tips.

Obligatory disclaimer: While we’ve collected tips and pointers below, and we encourage webmasters to “please try the following at home,” this is by no means an exhaustive list for your website’s security. We hope it’s useful, but we recommend that you conduct more thorough research as well.

• Check your server configuration.

• Stay up-to-date with the latest software updates and patches.

• Regularly keep an eye on your log files.

• Check your site for common vulnerabilities.

• Be wary of third-party content providers.

• Try a Google site: search to see what’s indexed.

• Use Google’s Webmaster Tools.

• Use secure protocols.
  

• Read the Google Online Security Blog.

Here’s some great content about online security and safety with pointers to lots of useful resources. It’s a good one to add to your Google Reader feeds.

• Contact your hosting company for support.

We hope you find these tips helpful. If you have some of your own tips you’d like to share, feel free to leave a comment below or start a discussion in the Google Webmaster Help group. Practice safe webmastering!

How can URL parameters, like session IDs or tracking IDs, cause duplicate content?
When user and/or tracking information is stored through URL parameters, duplicate content can arise because the same page is accessible through numerous URLs. It’s what Adam Lasnik referred to in “Deftly Dealing with Duplicate Content” as “store items shown (and — worse yet — linked) via multiple distinct URLs.” In the example below, URL parameters create three URLs which access the same product page.

Why should you care?
When search engines crawl identical content through varied URLs, there may be several negative effects:

How we help users and webmasters with duplicate content
We’ve designed algorithms to help prevent duplicate content from negatively affecting webmasters and the user experience.

If you find you have duplicate content as mentioned above, can you help search engines understand your site?
First, no worries, there are many sites on the web that utilize URL parameters and for valid reasons. But yes, you can help reduce potential problems for search engines by:

How can you design your site to reduce duplicate content?
Because of the way Google handles duplicate content, webmasters need not be overly concerned with the loss of link popularity or loss of PageRank due to duplication. However, to reduce duplicate content more broadly, we suggest:

If you follow this guideline, your webserver logs could appear as:

And the session file storing the raw cookie information may look like:

How can we better assist you in the future?
We recently published ideas from SMX Advanced on how search engines can help webmasters with duplicate content. If you have an opinion on the topic, please join our conversation in the Webmaster Help Group (we’ve already started the thread).

We launched Webmaster Central back in August 2006, with a goal of creating a place for you to learn more about Google’s crawling and indexing of websites, and to offer tools for submitting sitemaps and other content. Given all of your requests and recommendations, we’ve also been busy working behind the scenes to roll out exciting new features for Webmaster Tools, like internal/external links data and the Message Center, over the past year.

And so today, we’re unveiling a new look on the Webmaster Central landing page at http://www.google.com/webmasters. You’ll still find all of the tools and resources you’ve come to love like our Webmaster Blog and discussion group — but now, in addition to these, we’ve added a few more you might enjoy and find useful. We hope that the new layout will make it easier to discover some additional resources that will help you learn even more about how to improve traffic to your site, submit content to Google, and enhance your site’s functionality.

Here’s a brief look at some of the new additions:

We’d love to get your feedback on the new site. Feel free to comment below, or join our discussion group.

First, a brief recap: In late 2005, we started emailing webmasters to let them know that their site is violating our Webmaster Guidelines and that we have temporarily removed some of their pages from our index. A few months ago we put these emails on hold due to a number of spoofed messages being sent from outside Google, primarily to German webmasters. Then, in mid-July, we launched Message Center in our webmaster console, which allows us to send messages to verified site owners.

While Message Center is great for verified site owners, it doesn’t allow us to notify webmasters who aren’t registered in Google’s Webmaster Tools. For this reason, we plan to resume sending emails in addition to the Message Center notifications. Please note that, as before, our emails will not include attachments. Currently, the Message Center won’t keep messages waiting if you haven’t previously registered, but we hope to add that feature in the next few months. We’ll keep you posted as things change.

I’m happy to announce that Webmaster Tools is expanding support for webmasters outside of the English-speaking world, by supporting Internationalizing Domain Names in Applications (IDNA). IDNA provides a way for site owners to have domains that go beyond the domain name system’s limitations of English letters and numbers. Prior to IDNA, Internet host names could only be in the 26 letters of the English alphabet, the numbers 0-9, and the hyphen character. With IDNA support, you’ll now be able to add your sites that use other character sets, and organize them easily on your Webmaster Tools Dashboard.

Let’s say you wanted to add http://北京大学.cn/ (Peking University) to your Webmaster Tools account before we launched IDNA support. If you typed that in to the “Add Site” box, you’d get back an error message that looks like this:

Some webmasters discovered a workaround. Internally, IDNA converts nicely encoded http://北京大学.cn/ to a format called Punycode, which looks like http://xn--1lq90ic7fzpc.cn/. This allowed them to diagnose and view information about their site, but it looked pretty ugly. Also, if they had more than one IDNA site, you can imagine it would be pretty hard to tell them apart.

Since we now support IDNA throughout Webmaster Tools, all you need to do is type in the name of your site, and we will add it correctly. Here is what it looks like if you attempt to add http://北京大学.cn/ to your account:

If you are one of the webmasters who discovered the workaround previously (i.e., you have had sites listed in your account that look like http://xn--1lq90ic7fzpc.cn/), those sites will now automatically display correctly.

We’d love to hear your questions and feedback on this new feature; you can write a comment below or post in the Google Webmaster Tools section of our Webmaster Help Group. We’d also appreciate suggestions for other ways we can improve our international support.

Reporting is better, too. You’ll now be told of multiple problems per line if they exist, unlike earlier versions which only reported the first problem encountered. And we’ve made other general improvements to analysis and validation.

Imagine that you’re responsible for the domain www.example.com and you want search engines to index everything on your site, except for your /images folder. You also want to make sure your sitemap gets noticed, so you save the following as your robots.txt file:

You visit Webmaster Central to test your site against the robots.txtanalysis tool using these two test URLs:

Earlier versions of the tool would have reported this:

The improved version tells you more about that robots.txt file:

We also want to make sure you’ve heard about the new unavailable_after META
tag announced by Dan Crow on the Official Google Blog a few weeks ago. This allows for a more dynamic relationship between your site and Googlebot. Just think, with www.example.com, any time you have a temporarily available news story or limited offer sale or promotion page, you can specify the exact date and time you want specific
pages to stop being crawled and indexed.

Let’s assume you are running a promotion that expires at the end of 2007. In the headers of page www.example.com/2007promotion.html, you would use the following:

<META NAME=”GOOGLEBOT”
CONTENT=”unavailable_after: 31-Dec-2007 23:59:59 EST”>

The second exciting news: the new X-Robots-Tag directive, which adds Robots Exclusion Protocol (REP) META tag support for non-HTML pages! Finally, you can have the same control over your videos, spreadsheets, and other indexed file types. Using the example above, let’s say your promotion page is in PDF format. For www.example.com/2007promotion.pdf, you would use the following:

X-Robots-Tag: unavailable_after: 31 Dec
2007 23:59:59 EST

Remember, REP META tags can be useful for implementing noarchive, nosnippet, and now unavailable_after tags for page-level instruction, as opposed to robots.txt, which is controlled at the domain root. We get requests from bloggers and webmasters for these features, so enjoy. If you have other suggestions, keep them coming. Any questions? Please ask them in the Webmaster Help Group.

In the past year, the number of sites affected by malware/badware grew from a handful a week to thousands per week. We noted your suggestions to improve communication for webmasters of affected sites — suggestions mentioned in our earlier blog post “About badware warnings” as well as the stopbadware discussion group. Now, Webmaster Tools provides malware reviews.

If you find that your site is affected by malware, either through malware-labeled search results or in the summary for your site in Webmaster Tools, we’ve streamlined the process to review your site and return it malware-label-free in our search results:

1. View a sample of the dangerous URLs on your site in Webmaster Tools.
2. Make any necessary changes to your site according to StopBadware.org’s Security tips.
3. New: Request a malware review from Google and we’ll evaluate your site.
4. New: Check the status of your review.
• If we feel the site is still harmful, we’ll provide an updated list of remaining dangerous URLs
• If we’ve determined the site to be clean, you can expect removal of malware messages in the near future (usually within 24 hours).

We encourage all webmasters to become familiar with Stopbadware’s malware prevention tips. If you have additional questions, please review our documentation or post to the discussion group. We hope you find this new feature in Webmaster Tools useful in discovering and fixing any malware-related problems, and thanks for your diligence for awareness and prevention of malware.

Held on June 27th, Searchnomics 2007 gave us (Greg Grothaus and Shashi Thakur) a chance to meet webmasters and answer some of their questions. As we’re both engineers focused on improving search quality, the feedback was extremely valuable. Here’s our take on the conference and a recap of some of what we talked about there.

Shashi: While I’ve worked at Google for over a year, this was my first time speaking at a conference. I spoke on the “Search Engine Friendly Design” panel. The exchanges were hugely valuable, helping me grasp some of the concerns of webmasters. Greg and I thought it would be valuable to share our responses to a few questions:

Does location of server matter? I use a .com domain but my content is for customers in the UK.

I have many different sites. Can I cross-link between them?

Greg: Like Shashi, this was also my first opportunity to speak at a conference as a Googler. It was refreshing to hear feedback from the people who use the software we work every day to perfect. The session also underscored the argument that we’re just at the beginning of search and have a long way to go. I spoke on the subject of Web 2.0 technologies. It was clear that many people are intimidated by the challenges of building a Web 2.0 site with respect to search engines. We understand these concerns. You should expect see more feedback from us on this subject, both at conferences and through our blog.

Any special guidance for DHTML/AJAX/Flash documents?

When Google originally introduced Supplemental Results in 2003, our main web index had billions of web pages. The supplemental index made it possible to index even more web pages and, just like our main web index, make this content available when generating relevant search results for user queries. This was especially useful for queries that did not return many results from the main web index, and for these the supplemental index allowed us to query even more web pages. The fewer constraints we’re able to place on sites we crawl for the supplemental index means that web pages that are not in the main web index could be included in the supplemental. These are often pages with lower PageRank or those with more complex URLs. Thus the supplemental index (read more – and here’s Matt’s talk about it on video) serves a very important purpose: to index as much of the relevant content that we crawl as possible.

The changes we make must focus on improving the search experience for our users. Since 2006, we’ve completely overhauled the system that crawls and indexes supplemental results. The current system provides deeper and more continuous indexing. Additionally, we are indexing URLs with more parameters and are continuing to place fewer restrictions on the sites we crawl. As a result, Supplemental Results are fresher and more comprehensive than ever. We’re also working towards showing more Supplemental Results by ensuring that every query is able to search the supplemental index, and expect to roll this out over the course of the summer.

The distinction between the main and the supplemental index is therefore continuing to narrow. Given all the progress that we’ve been able to make so far, and thinking ahead to future improvements, we’ve decided to stop labeling these URLs as “Supplemental Results.” Of course, you will continue to benefit from Google’s supplemental index being deeper and fresher.