But the members who counted on ULI’s website weren’t always able to find what they were looking for.  ULI’s approximate 60 websites had different implementations (many not on Sitecore) and the current search did not effectively allow the user to search across all ULI related sites. Beaconfire implemented the Google GSA search application, giving ULI the robust functionality of Google Search. Part of this implementation included giving the user an option to search on the specific site or across all ULI and created a common advanced search and search results page across all ULI sites.  Beaconfire also implemented a number of Sitecore updates which allowed the search results to display certain data, such as secured pages and the update date.   You can try out the new search functionality at  http://www.uli.org/ or http://urbanland.uli.org/.

Now, the ULI websites offer a consistent and reliable search experience and results for ULI members and site users.

Google just announce their rollout of the new +1 button, a way for searchers to promote links to their Google connections.  Instead of just sharing cool links on Facebook and Twitter, Google wants you to start marking useful links in your search results, so your friends can see them.

The web is a-buzz with criticism of Google’s latest foray into social networking, and whether it will be more successful than Buzz. End users don’t need to worry about it yet: it’s being rolled out slowly, and you need to create a public profile before you can use it. But, if you’re a webmaster or do online marketing, you should be aware of how it might affect you:

• Your +1′s will affect your organic search rankings – they will be added as a new factor in Google’s ranking algorithms. Just as external links to your site are important to cultivate, you should cultivate +1′s as well.
• +1′s may appear next to your Adwords ads as well as organic search results – another way for ads to help boost your influence.
• There isn’t one yet (at least not officially), but Google will soon release a +1 button that can live on your website, next to your “like” and “tweet” buttons. This may be the most common place that people “+1″ your content, since I would imagine most people won’t recommend content from search results if they haven’t visited it yet.

This move from Google has been a long time coming, as they try to find a way to bring social recommendations into their standard results, and to compete with Facebook as the place that web users go for interesting content.  We’ll have to wait and see whether it takes off, but to me, the fact that +1 is tied to search rank means that we’ll at least start seeing the button on most websites. Whether people will use it is another question.

Can you see yourself using +1 as you browse the web?  Do you think you’ll add it to your site? If you’ve tried it out already, I’d love to hear what you think so far!

1. Encourage links from supporters

One of the most important factors for SEO is having incoming links from reputable external web sites. As a non-profit, you have an advantage in this area, because you likely have supporters who are happy to promote your cause with a link. Take advantage of this — if your organization has a blog or makes announcements online, make sure that content is easily shareable. Provide logos, badges, or widgets that your supporters can easily plug into their own web sites.

Beware of linked badges/graphics that don’t give search engines enough information. From NTEN’s Blog:

Badges are an excellent way to get your supporters linking to you, but too many nonprofits make a common mistake: they fail to incorporate text links into their badges. Search engines prefer text links to graphics, as they signal which keywords are related to the linked page. (Remember, search engines can’t read graphics. Yet.) So, ask your supporters for links and be sure to provide them with SEO-friendly badges.

2. Don’t forget about pages hosted elsewhere

If your donation form isn't hosted on your site, it won't appear in your Google Sitelinks.

The pages stored in your content management system may already be search engine optimized — but if some of your most important pages are hosted elsewhere (through your CRM system, for example), they may not be optimized in the same way.

• The externally hosted page may have a different domain name than the rest of your site, which makes it difficult for search engines to associate it with your organization. For example, a donation form hosted on a separate domain cannot be listed as one of your Google Sitelinks. However, if you create a “Donate” page on your own site that directs users to your donation form, that can appear as a Google Sitelink.
• You may not have as much control over the format, URL, title, heading usage, and meta tags for pages hosted elsewhere. Take the time to identify workarounds or dig deeper into the customization options for your highest priority pages.

3. Evaluate the calls-to-action throughout your site

• Identify the pages on your site that already receive plenty of traffic from search engines, then make sure your most important calls-to-action are featured or at least accessible from those pages.
• Make your calls-to-action SEO friendly by incorporating text links containing strong keywords and using alt tags for banners/images.

4. Manage your online identity

• Which is more recognizable to your target audience: your organization’s full name or its acronym? Or maybe neither — are people most likely to be searching for a well-publicized project or campaign you’ve run recently? Does it depend on the audience? Account for these variations in your SEO strategy.
• Be sure that your website is listed where it should be, both in broad non-profit listings sites (Idealist, Charity Navigator) and in topic or geographically-focused resources.

5. Do your keyword homework

• When branding new campaigns and projects, do keyword research. Does the title of your new campaign align with the terms that your audience is searching for?
• Identify the keywords that are most specific to your cause and be able to differentiate yourself.

Do you have other SEO ideas that non-profits should have on their radar? Post them in the comments.

But how do you decide which keywords to focus in on?

Here are a few free tools to help you out:

• Google Analytics / Webmaster Tools
  Understand what keywords are driving your site traffic today

  If you use Google Analytics, check out the following reports:

  • Traffic Sources > Keywords will show you keywords used on external search engines that lead visitors to your site, then show you what happens once they enter your site.
  • Content > Site Search > Search Terms will show you keywords used for your internal search, which are often not specific enough to target as keywords for public search engines, but will still give you valuable insight into your visitors’ goals and vocabulary.

  And in Google Webmaster Tools:

  • Your Site on the web > Search queries is similar to the Google Analytics’ Traffic Sources > Keywords report, but reveals more about what’s happening on the search engine side of the equation. How far down on the page does your page appear? What is the click through rate?

  Nice perk: Both of these tools make it easy to monitor keyword performance over time. In Webmaster Tools, you can “star” certain terms that you want to keep an eye on. In Analytics, you can set up reports to arrive in your email inbox as a reminder to check back periodically.

  Also see: If your CMS provides reporting, check for reports similar to these.

• Google Adwords Keyword Tool
  Understand the larger keyword universe, and how you fit into it

  Don’t be misled by the title: Even if you aren’t using Adwords, you can benefit from this tool. For any keywords you enter, it will return the relative popularity of the phrase (in # of searches) and give you an idea of how competitive it is. It will also give you tons of additional ideas to add to your list.

  Nice perk: You can download the data as a spreadsheet and work with it offline.

  Also see: WordTracker, WordStream
  

• SEOMoz’s Term Extractor
  Analyze specific pages of your site
  

  Use this tool to get the search engine’s view of important keywords on a specific page. Do they match the keywords you’re targeting? If not, you can make changes to page elements (Title, URL, heading tags, and alt tags, among others), then try again until you are satisfied with the relative weight of the keywords for the page. This is a great before-and-after test as you optimize pages throughout your site.

  Nice perk: You can add this as a browser button/bookmark to analyze the page you’re currently browsing with a single click.

  Also see: If you use Google Webmaster Tools,  go to Your site on the web > Keywords. This report gives you the equivalent of the Term Extractor tool, but on a site-wide rather than page-specific level.

And lastly, if I lost you at “if you’ve read anything about search engine optimization,” Google’s SEO Beginner’s Guide (PDF) is a good place to start.

Google Webmaster Tools is a free service provided by Google that will help you to see the “big picture” of how your site exists on the web and how you can improve both its presence and performance. To get started, log in with any Google Account. You’ll need to add your site and complete a brief verification process. Then you’re ready to go.

Here are a few things you can accomplish through Webmaster Tools:

Learn about Incoming Traffic

Understanding the intentions of incoming visitors will help you to make sure they have a positive experience on your site.

• View Top Search Queries shows when and how your web site is appearing in Google search results. Which keywords display your site the most? Which result in click-throughs? Learning the answers to these questions (and identifying the differences between them) will help you to ensure that your site’s content meets the needs of incoming traffic. If certain keywords are missing from the list or ranked lower than you would expect, you may need to add more content to your site related to that keyword and ensure that others are linking to it.

  More highlights after the jump.

• Links to Your Site shows which pages on your site are being linked to from other sites, and how many visits these links are generating. Why is this important? Maybe one of your most linked-to pages is in desperate need of an overhaul. Or maybe you’ll discover that most new visitors are not entering your site through the homepage, so they’re missing the important announcement you’ve posted there. Links to Your Site will also show you which sites visitors are coming from, which will give you even more insight into what they’re looking for.
• Keywords is a list of the most common keywords found by Google when crawling your site. Ideally, this list should reflect your site’s content. If it doesn’t, you should confirm that your site is being crawled successfully and then evaluate your content strategy and use of keywords.

Guide Google Through Your Site

Prevent Google from indexing specific pages and direct it to the ones that matter.

• Sitemaps will allow you to submit a sitemap to Google and detect problems associated with it. You won’t need to re-submit the map every time you change it, but it is worth revisiting this page to catch any problems that Google detects.
• Crawler access guides you through the process of creating and testing a robots.txt file, which tells search engines which pages or directories of your site to ignore.

Catch Problems Early

By monitoring your site proactively, you can avoid problems that result in unhappy visitors or losses in traffic.

• Site Performance (found under Labs) will show you how fast your site’s pages load and how this compares to other sites. It provides a high-level timeline chart, a list of specific pages and their load times, and then gives tips on how to speed up those pages.
• HTML Suggestions (found under Diagnostics) will identify duplicate or missing page titles and meta descriptions, which both have an impact on how visitors find your page through Google search. What makes a good page title and description? Google provides some tips on that, too.

Although these are some good starting points, this isn’t an exhaustive list of everything available in Webmaster Tools. To keep up with the latest features and best practices, visit the Google Webmaster Blog.

What I do know about is what knowledge is online. Since Sir Tim Berners-Lee (Yes. He was knighted.) created the World Wide Web to link documents together on the then nascent Internet, knowledge became more than monolithic documents or books that were linked loosely via citations and references. Instead of specifying in APA, MLA, Chicago, or Turabian style where the source of a particular knowledge was, one could directly link it using something called “HyperText”, or what some know as “Hyper Text Markup Language”. Today, all websites that you see online are built with a combination of HTML, some JavaScript, and possibly some Flash or Java.

Never mind what the knowledge is built with. Let’s examine very briefly the trend of tagging and keywording content. “Tagging” or “keywording” existing content can help web site users browse through indexes of such tags or keywords similar to how a book may be indexed by the editors and publishers. What was the first representation of such archaic systems on the Internet? Search Engines. Search engines were of two varieties. Directories such as Yahoo or Excite and Search Indexes such as Alta Vista. Both had their place. If you wanted to browse through a human edited directory, you had good reason to. People had taken the time to classify and organize the websites using their God given intellect. If you wanted to search, you also had good reason to.

Since the computer was able to search all of the content in a set of knowledge automatically, search indexes could be created using complex algorithms. One of the reasons Google succeeded in the search indexing industry was because they figured that relevant pages would link to each other and the more relevant websites would have higher density of connections. Similar sites around similar sets of knowledge would coalesce using the network effect to create centers of knowledge. Further more, content that is properly tagged and keyworded can heighten the quality of the search because the search engine has the power to correlate keyword usage and the keywords themselves.

What is OpenCalais?

The Calais Web Service:  The Calais web service automatically attaches rich semantic metadata to the content you submit. Using natural language processing, machine learning and other methods, Calais categorizes and links your document with entities (people, places, organizations, etc.), facts (person “x” works for company “y”), and events (person “z” was appointed chairman of company “y” on date “x”). (OpenCalais 2009)

Now that you have a bird’s eye view of how knowledge is stored and sought online, let’s talk about OpenCalais. OpenCalais is a toolkit that can be used by Content Management systems to automatically find the best set of keywords to represent a particular piece of content. Instead of having to pour your blood, toil, sweat, and tears, you can have a machine do it for you. OpenCalais is not for consumers and must be integrated into an existing piece of online content management software, or be integrated into a custom solution.
If you have a 5-10 page website, you probably don’t need to use OpenCalais. If you have a catalog of any kind, be it books, products, article, or news you should look into integrating this tool so that your website becomes easier to search and navigate on your website as well as through external search engines. OpenCalais looks at your content and finds out if there are references to people, cities, companies, industry terms, and a number of other existing knowledge centers.

A test using the OpenCalais Document Viewer

To demonstrate what OpenCalais can extract from your content, I submitted the content of this article above this sentence to get this screenshot. I’m only showing the highlight of one found term. Similarly, the other terms that were found are connected to vast amounts of knowledge already exist intent on the internet. Those knowledge centers have relevant information and hence if your content is relevant, it is given a high content relevancy rating for each word that it finds. Apparently, my article on “Online Knowledge” only gets a rating of 35%. It’s better than nothing. I’ll try again next time. You can try yourself using their Document Viewer. If you want help in implementing this great tool, let us know by contacting us through our website’s Contact Us page.

A website’s search can be similar.  Many clients we talk to are convinced that their site search doesn’t work and needs to be torn out — and sometimes that’s true.   We can gut the whole works, put in a new Google Mini or MS Search Server, spend hours dividing your site into collections, setting up separate searches based on audiences or site sections, but that’s not always necessary.  As a counterpart to the website facelift, the site search facelift seeks the same result: more for less.  How? By applying lessons from your search metrics, particularly what the top search terms of people seeking information at your site are and then making specific corrections to content.

While a site search facelift has several techniques to improve results, one of the easiest steps you can take to improve search results is to review frequent search terms and provide recommended links through your search engine for those terms.  The term used is dependent on the search engine, but is commonly referred to as “best bets” or “suggested results”.

Perhaps your search engine doesn’t make those frequently searched for terms reports available? Don’t have that data in the first place? Don’t be so sure – if you’re using Google analytics, most of the time that information is available to you irrespective of your search engine technology.

The best thing about this method is that it generally follows the 80/20 rule where 80% of your searches are coming from 20% of the search terms people use.  Start with the terms most frequently searched form and you are automatically handling the search results in the order that will gain the most benefit from clear, recommended content suggestions.  And while there’s a lot more to a full search facelift, this is a quick way to get some significant results.

With the recent agreement between Yahoo and Microsoft making MS Bing the search platform for all of Yahoo’s sites, there’s now a new wrinkle in the SEO game.  Take the two numbers in your referral logs for Bing and Yahoo, combine them together — big enough to matter now? Especially with MS pushing the technology everywhere it can?  Right now you’ll continue to have traffic from both Bing and Yahoo, but moving forward, most sites will only have to optimize for a single additional search engine in the #2 position — Bing.

If you decide Bing is worth watching, try out these tips on optimization.  Remember – SEO is always a bit of black magic rather than hared science, but the rewards can definitely be worth the effort.

Shortening urls provides some important benefits to the person sharing the link, which is why they’ve become so popular.  Aside from taking up less space (thus making them more tweetable), many of the services provide tracking data, giving you details about how your content is being used.  That can be really fun for the casual user, and valuable for organizations.  Some people think they look tidier than long urls.

But there are two (or maybe three) other parties affected by the shortened url, and they aren’t so well served by it.

The reader, who comes across a shortened url in a post or tweet, may have no idea where the link is supposed to go.  All they get is the context of the link, which may be as little as “check out this cool link”.  Without clicking on it, they have no way to know where it goes.  (Not totally true – there exist Firefox plugins that will expand them for you – but it is true for most users.)  From a shortened url, you can’t tell whether the destination is the New York Times, a non-profit, a personal blog, or something far less reputable.  For you, as a user, all the clues you use to decide whether to click on a link may be missing.  That’s a problem for the person posting the content – if you don’t trust them, you’re much less likely to click.  Furthermore, it’s less likely to catch your interest.

The benefit of longer, SEO-conscious urls is that you should be able to tell what you’ll find before you click: you know the domain, the page name, and possibly other information.  Take this post, for example: http://www.beaconfire.com/blog/2009/05/26/url-shorteners-how-to-stay-out-of-trouble tells you that a company called Beaconfire is providing the content (even if you’re not familiar with who we are), that it’s a post from our blog, that it was published in May 2009, and that the topic is url shorteners.  Armed with that information, you can decide whether you’re interested enough to click on it.  Run it through a service like bit.ly, and all that information is lost – but at least the link is short?

There’s also the content provider – the site that’s being linked to.  At the very least, they’re losing their branding, since their domain is no longer displayed.  A lot of SEO experts are also concerned that the shortening services themselves – which parse the link and redirect you to the destination – may rob the destination content of its search ranking, that search bots may not follow the link and give credit to the destination.  Is this a serious concern?  It’s an open question, and every service works a little bit differently.  Some do much better than others, so it’s worth doing your homework on which url shortener will best preserve your SEO.  (In all cases, you lose the benefit of having keywords in the link text, which is typically just the url.)

The third party affected by shortened urls is the internet as a whole.  Joshua Schachter has written a fantastic and detailed explanation of why shortened urls are hurting the web – the gist is that these services are middlemen in an already complex system.  They are numerous, fragile, and unreliable, and if they someday disappear, we’ll have a lot of links that no longer point to their content, and the internet will be broken.  If that seems like an overly philosophical concern, consider how quickly sites often come and go on the internet.

If it were up to me, I’d say these services aren’t worth the trouble.  But, unless the whole world wakes up tomorrow and decides that Twitter is just no longer cool, I think shortened urls will be around for a while.

So, how can you embrace social media without the downsides of shortened urls?

Most importantly, don’t use them when you don’t need them.  If you’re twittering to promote your own campaign, create a friendly url that will fit easily in a tweet, and you’ll promote recognition for your site.  This is especially true if you’re not using any tracking features from the service.

If you do need to use them, look for a service that lets you customize the url to some extent, as well as following good SEO practices with their redirects.  Check this list for an easy feature comparison. Be sure to provide plenty of context about why users should click through.

One service, kl.am, even lets you append Google Analytics campaign variables for your short url so you can track them by source in Analytics.  I can’t speak to kl.am’s other features, but this seems like a nice one – often, appending source codes is part of what makes urls long and unsightly.

The biggest lesson, to me, is not to use shortened urls frivolously.  If you don’t need them, don’t use them.  Period.  If you do need to keep your url short, consider your readers and your brand, and use these services wisely.

Right now, we’re in the midst of a website redesign that utilizes Search Server 2008. It’s a pretty common type of redesign for us at Beaconfire – a website built on a content management system that doesn’t have a built-in search technology, or at least not one we want to use. The client’s search needs are also pretty typical for a lot of site builds:

1. The search is primarily targeted at the site being redesigned – federated searching of external content might be a plus but takes a back seat to getting good search results for the site the user is searching on.
2. The focus in on public HTML content, not documents or database items, and there’s little in the way of heavily role-based content restrictions
3. The need for advanced filtering is minimal but needs to be expandable for the future
4. The search is fully integrated into the main website. The search and the results returned are displayed seamlessly in pages on the site instead of shuttling users off to another server.
5. Emphasis is placed on returning the best results possible through keywords and “best bet” mechanisms

Those requirements result in what is decidedly not an Enterprise level of search, but a very good example of the type of search plenty of customers want in a redesign – not a fancy search, just a particularly smooth and relevant one.

With those requirements in mind, here’s the my quick take on how the two technologies stack-up based on my impressions with both, right after the jump.

Price:

A quick and easy win for Microsoft. You can’t beat free, and the circumstance where you need to buy a license (load-balanced search servers) is uncommon enough that those who need it can afford the minor software fee. For many organizations, the Google Mini’s rather reasonable cost to purchase the box ($2999 for the standard model with 2 years of support) is less of a burden than the hosting costs associated with racking it up.

Administering Indexing:

I’ve had far too much experience with Minis to still be as baffled as I am by its indexing interface. It’s only become more complex over time, with various settings in different places for toggling the frequency of particular URLS to get indexed (which only applies in certain crawling modes), the feature to tell it to index a page as soon as possible (as in NOW) without seeing much effect, and the predictive ability to tell you what it plans to index in the next hour that rarely seems to match reality. Starting a Google crawl is like a strongman competition for pulling a dump truck with a nylon rope and your own teeth – incredibly tough to get started, slow going at the best of times, and very painful to stop. MS Search, on the other hand, is brilliant –check a box and it starts, crawling the site extremely quickly.

Development to Deployment:

Search engines are a particular pain to network because they are usually within the same network segment as the website you’re deploying, yet they are set to spider content as if they were an external user looking at your site. Combine that with the issue of switching a DNS entry on a redesign launch and you get a ton of small but significant problems related to hostnames and networking. Every time we launch, there are tons of tasks for switching the search settings, reindexing immediately and testing. Search 2008 makes it a lot easier with a simple mechanism – integrated URL mapping. Just replace your staged website with the URL of the soon-to-be-live site and you can worry about reindexing later, when things aren’t so crazy. Google, take note – I want this in the Mini.

Reporting

Microsoft’s approach is simple – a bunch of reasonable, pre-made reports. While Google has a couple prepared reports like content type stats, most of the appliance uses a byzantine method of creating a report, naming it, refreshing the page to see that it already finished by the time you named it, and then viewing, is terribly unfriendly by comparison.

Result Summaries

Given that we do full integrations of both search interface and results display into our site pages, we want a result set that can be easily shoehorned into whatever interface the designers want, from basic results to AJAX-y showpieces. Both engines return text summaries that show the term hits in the context of the page and string it all together into a hit highlighted abstract. Google’s hit highlighting is basic, but good enough for most purposes, as it surrounds hit terms with <b> tags – like a lot of the HTML Google produces, it’s lowest common denominator (as anyone who has ever looked at their <FONT> tag ridden XSLT stylesheets can attest to). Microsoft uses a numbered hit tag system (c0, c1, etc), perhaps easy enough to understand once various helpful bloggers explain it, but still a bad idea. Note to Microsoft – don’t return your hit results with tags that Firefox and Chrome can style natively with CSS, but your own Internet Explorer ignores, or requires an XSL transformation when a span with classes or ids would do fine, and don’t number the tag names so each one has to be enumerated separately which is why you typically don’t see tags with numbers in the name. Devs, be prepared to do some wholesale text replacement on Search 2008 results if you want to format them nicely, like placing ellipses between term hits.

Meta Data

A feature list examination would give Microsoft a clear edge here –defined crawled properties for many different types of documents, which can be interactively mapped into managed properties in a chosen order, so properties can gracefully fall back onto less relevant ones if the most targeted ones aren’t available. That’s fabulous, especially compared to Google’s need-to-know approach of “we’ll tell you what metadata properties we indexed when you search and not before.” But Google is also understanding of some of the basics – just about anything you index is going to be a string, except for numbers and dates. There’s a good chance you’re going to want to use dates for range filtering or sorting, and Google is also extremely tolerant at guessing dates from a variety of formats sources like meta fields or server headers. MS Search 2008, on the other hand, seems to think just about every HTML property is a text field, and won’t let you map crawled text properties to date type managed properties, simultaneously segregating HTML content away from nicely indexed documents like PDFs or Word files and forcing workarounds for dealing with ranges.

Indexing Redux:

You might recall I loved the instant gratification of the MS Search indexing. Less lovable is the fact that Search Server will happily replace a perfectly good index with whatever it just saw in its latest spidering attempt, even if it found absolutely nothing because, say, your entire site was down for a few minutes or some networking hiccup prevented the search from seeing your site. I’d love to say the Mini handles this situation very well – older versions had prominent toggles for a minimum number of crawled pages and specific page urls that must have been crawled before placing a staged index into production – but this feature is either gone or extremely well hidden in modern Minis.

Best Bets, Keywords and Synonyms

On the surface, both engines seem capable in this regard, and each has some unique features. Both have keymatching or “best bet” functionality, both allow synonyms but not fancy stemming through fuzzy logic. MS lets you specify authoritative pages. Google lets you import keymatches and related queries from text documents. Etc. Best bets (defining a single “featured” link for a common search term) and synonyms (equating a search for one term to be the same as another term) are particularly useful in the scenario where organizations want to push users to special events or articles. Unfortunately Search Server 2008 has a fine-print limitation on their synonym feature – synonyms are only relevant when coupled with an already existing best bet match, meaning that it’s basically just a way to keep from having to type the same best bet over again with a different term. I’d really love to see a true synonym system similar to Google’s.

Error Handling

To my mind, Google has this right – it is tolerant of mistakes or “bad” queries to the point that it’s nearly impossible to cause an actual error. The Microsoft querying web service has the heavy handed approach of throwing an exception for just about everything – from an unrecoverable network problem, to a developer configuration error when requesting a property the search doesn’t know about, to a user search of something like the words “and the”. That last query returns “Your query included only common words and / or characters, which were removed. No results are available. Try to add query terms. ” They could have added “Keep trying at that internet thing, you’ll figure it out eventually.” Yes, each of those conditions should be handled, but they’re not remotely in the same class. Developer exception trapping shouldn’t be a substitute for reasonable behavior from a search engine.

Results and Ranking

This is the most crucial feature, and in my opinion, the Mini wins this hand down. Google’s search technology is widespread for a very good reason – their “magic” algorithms do a great job of returning relevant results. Things have only improved over time as they’ve added features to the Mini borrowed from its big brother Google.com search, like automatically filtering similar results to drastically reduce the number of results displayed. Meanwhile, MS lacks some important features like partial page indexing (excluding content within a page from being indexed) which results in matches for keywords found in common navigation areas, etc. Google has had those features (googleoff:index) forever but has improved the relevancy to the point where they hardly need to be implemented any more. Google has also begun opening up their sacrosanct black box of assigning hit relevance by letting administrators apply bias to results using date and site section as a factor.

So there’s my quick take. I’ve skipped over a lot of functionality, and there are areas like federated searching and role-based authentication where MS clearly shines. Then there is the full-fledged Google Search Appliance whose features I haven’t touched on. So tell me where I’m right, tell me where I missed things, and tell me where I’m just plain wrong in the comments.