Time and the Google sandbox

Finally, the one factor you have little control over. You only really have con- trol over time in the sense that the sooner you get started, the older your search-engine project becomes. Age is critical because the older the site, the more credibility the search engines give it.

There’s something known as the Google Sandbox or aging delay. (Some people will tell you that these are actually two different types of time-related effects.) The idea is that when Google first finds your site, it puts it into a sandbox; it may index it, but it won’t necessarily rank it well to begin with. It may take months before the site comes out of the sandbox. (People talk about the Google sandbox, but it seems likely that other search engines have something similar.)

There’s a lot of debate about the effect of age; some say it’s critical, and that for about eight months your site hasn’t a chance of ranking well (I’m not in that camp), and others say that while the search engines may take into account age to some degree, it’s by no means an overwhelming factor.

It comes down to this: The longer your domain has been registered, the better, and the longer your site has been up, the better. So you have control over this essential factor in just one way; the sooner you get started, the better. Register your domain name as soon as possible. Get a site, even a few pages, posted as soon as possible, and get links pointing from other sites to your site as soon as you can. Get new content posted as soon as possible. The sooner you start, the sooner you’ll start ranking well.

Reading history

We don’t know exactly how Google handles all this, of course, but you can be fairly sure that Google uses some kind of historical data to help rank pages. In fact there’s even a patent sub- mitted in the names of various Google employ- ees (though strangely, without Google’s name itself on the patent), that discusses the idea of using historical data. (A long complicated URL
 takes you to the patent, so I’ve provided the link at www.searchenginebulletin.com .) This document is wonderful bedtime reading, if you’re looking for a way to get to sleep without drugs. You won’t find an explanation of how Google ranks Web pages, but you will find a lot of interesting possibilities.

Determining Your Plan of Attack

Now you know what you’re facing. As you read in blog 1, you can more or less forget those thousands of search sites and focus on no more than five search systems. And as I explain in this blog, you have six essential fac- tors to play with: keywords, content, page optimization, links, submissions, and time.

Forget about time . . . all I’ll say is, get started right away! As for the other fac- tors, how do you proceed? It depends to some degree on your budget and the competitiveness of the area you’re working in.

Do a keyword analysis. Regardless of competition or budget, you have to do one. Would you study for an exam without knowing what the exam is about? Would you plan a big meal, and then send an assistant to the grocery store without explaining which supplies you need? If you don’t do a keyword analysis, you’re just guessing. In my experience, you’ll almost certainly fail to pick all the right keywords. See blog 5 for the lowdown on how to do this analysis.

Create readable pages. If you want your site to appear, you have to create pages that the search-engine spiders or bots can read. (This isn’t an issue for the search directories, but if you expect a bot to read your site, the pages have to be readable.) You might be surprised to hear that millions of pages on the Web cannot be read by search engines. For the lowdown on determining whether your pages are being read, see blog 2; to find out how to fix the problem if they’re not, see blogs 6 and 7.

Create keyworded pages. Having readable pages is just a start. Next you have to put the keywords into the pages — in the right places and in the right format. See blog 6 for details.

Register with the search systems. When your pages are ready to be indexed, you need to do two things:

• Let the search systems know where those pages are.

• Get the search systems to include the pages in their indexes and

directories.

Sometimes these tasks are harder than you might expect. You can get into the search systems various ways, as described in detail in blogs 11 and 12.

Get other sites to link to your site. Check out blogs 14 and 15 to find out how the number and type of links pointing to your site affect your rank.


The preceding strategies are the basics, but you may want to — or even need to — go further. I cover these additional techniques in detail later:

Register with other places. You may also want to register at specialized sites that are important for your particular business. See blog 13. Register with the shopping indexes. If you’re selling a product, it’s a good idea to register with the shopping indexes. Although these indexes don’t match the big search systems in volume of searches, they’re still important. This is covered in blog 16.

Use Pay Per Click. You can get noticed in the search engines two ways. You can use natural search — that is, get ranked in the search engines without paying — or you can use Pay Per Click. Many companies go straight to Pay Per Click, a system by which you get ranked well but pay each time someone clicks a link to your site. This is usually not a good idea (though sometimes it’s a great way to push a product temporarily, such as a special offer), but at some point, you may want to use Pay Per Click in addition to natural search; see blog 17.

But there’s more. If you’re in a very competitive market, you may want to really push two techniques:

Create large amounts of content. Make hundreds, perhaps thousands, of pages of content.

Go after links in a big way. You may need hundreds, perhaps thou- sands, of links to rank well if your competitors have done the same.

This blog provides an overview of the search-engine battle you’re about to join. Now it’s time to jump in and make it all happen, so blog 4 explains what search engines really like to see: Web sites that people on the Internet believe are really useful.


Making Your Site Useful and Visible


Understanding the basic rule of Web success Knowing why search engines like content

Making your site work for visitors and search engines

bviously, it’s important to create Web pages that search engines will read and index, pages that you hope will rank well for important key- words. But if you’re going to build a Web site, you need to step back and figure out what purpose the site should serve and how it can accomplish that purpose.

Creating a useful site is the key. Even if your sole aim is to sell a product online, the more useful the site is to visitors, the more successful it’s likely to be. Take Amazon.com, for instance. It certainly wasn’t the first online retailer of blogs and music, or any of the other products it offers. But one of Amazon’s real strengths is that it doesn’t just sell products; it’s a really useful site, in many ways:

It provides tons of information about the products it sells. The informa- tion is useful even if you don’t buy from Amazon.

You can save information for later. If you find a blog you’re interested in but don’t want to buy right now, save a link to it and come back next month, year, or five years from now.

Other site owners can become partners and make money by promoting Amazon.

Other businesses can easily sell their products through Amazon. You can read sample blogs, look at tables of contents, listen to snip- pets of music, and so on.

You can read product reviews from both professional reviewers and consumers.

Would Amazon be so successful if it just provided lists of the products it sells, rather than offering visitors a veritable cornucopia of useful stuff? Absolutely not.

Consider this: The more useful your site is, the greater the chance of success. The more people talk about your site, the more likely journalists are to write about it, the more likely it is to be mentioned on radio or TV, the more people will link to it from their Web sites. Search-engine marketing and non-search- engine marketing are both important because either form of Web site promo- tion can lead to more links pointing to your site. And, as you find out in blogs 14 and 15, links to your site are critical to search-engine success.

With that in mind, this blog focuses on the basics about what you need to do to create a successful Web site.

Revealing the Secret but Essential Rule of Web Success

Here’s a simple rule to success on the Web:

Make your site useful and then tell people about it.

That’s not so complicated, really. Figure out how your site can be useful and then find as many ways as possible to let people know about it. You’ll use the search engines, of course, but you should be using other methods, too. Remember, the search engines are not the only way to get people to your site. In fact, many Web sites have succeeded without using the search engines as their primary method of attracting visitors.

Amazon — Success sans search

It’s unlikely that search engines were a large factor in Amazon’s success — Amazon grew rapidly mainly because of the enormous press attention it received, beginning in 1994. Today, I’d bet that relatively few people arrive at Amazon.com through the search engines. Rather, they already know the Amazon brand and go straight to the site, or they go through
 the hundreds of thousands of Amazon affiliate sites. And up until recent years, Amazon made relatively little effort to actively generate search-engine traffic. I suspect they pay more attention to it now; certainly their new Statistically Improbable Phrases (SIPs) and Capitalized Phrases (CPs) lists on their blog pages help them in the search engines.

Many successful companies have done little or nothing to promote them- selves through the search engines, yet they still turn up at the top when you search for their products or services. Why? Because their other promotions have helped push them higher in the search engines, by creating thousands, even tens or hundreds of thousands, of links to them around the Internet.

The evolving, incorrect “secret”

Over the last decade, a number of popular ideas about what makes a success- ful Web site have been bandied around, and all were wrong to some degree. Here are some of those dated secrets to successful Web sites:

Links: When the Web began booming in 1994, it was all about links. You would hear in the press that the secret to a successful Web site was link- ing to other sites.

Cool: Then people started saying that the secret of success was to make your site cool. Cool sites were more entertaining and more likely to attract repeat visitors.

Community: Then people started talking about community; yeah, that’s the ticket! The secret to a successful Web site was creating a community where people could meet and chat with each other.

Content: Then, around 2000, people discovered that the secret was con- tent. By putting more stuff, particularly textual information, on your site, you could be more successful.

Specific one-size-fits-all secrets to success never make sense.

The most harmful of the preceding ideas was that your site had to be cool. This silly idea led to the expenditure of billions of dollars on useless but pretty Web sites, most of which (thankfully!) have since disappeared. Unfortunately, some of the it’s-all-about-cool crowd is still in the Web business and still convincing businesses to spend money on ridiculous, wasteful things such as Flash intros for their Web sites.

Uncovering the real secret

Ready to hear the real secret of site-creation success? Your Web site has to be useful. The problem with the secrets I just mentioned is that they’re too specific, leading people to build sites that were in many cases inappropriate. Sure, links are important to Yahoo!, but they’re much less so to the vast majority of Web sites. If you own an entertainment site, you may want to make it cool and entertaining. Certainly community can be an effective tool, but not every site has to have it. Content is very important, too — especially from a search-engine perspective — but many successful Web sites don’t have much content. (I talk in more detail about content in the next section because it’s a special case.)

I’ve been writing this since 1997: Forget cool; think useful.

When you’re planning your Web site, think about what kinds of folks you want to attract to the site. Then try to come up with ideas about what fea- tures and information might be useful to them. Your site may end up with a lot of link pages, providing a directory of sorts for people in your industry. Or maybe you really need a cool and entertaining site. Or perhaps you decide to use discussion groups and chat rooms as a way to build community and pull the crowds into your site; that’s fine. Or maybe you decide to create a huge repository of information to attract a particular type of customer. That’s okay, too. Maybe you do all these things. But the important first step is to think about what you can do to make your site more useful.

Showing a bias for content

Content is a special case. Search engines are biased toward ranking content- heavy Web sites well for a couple of reasons:

Search engines were originally academic research tools designed to find text information. Search engines mostly index text — content.

Search engines need something to base their judgments on. When you type a term into a search engine, it looks for the words you provided. So a Web site built with few words is at a disadvantage right from the start.

As you discover elsewhere in this blog — such as in the discussion of PageRank in blog 14 — search engines do have other criteria for deciding if a Web site matches a particular search (most notably the number and type of links pointing to the site). But search engines do have a huge bias toward textual content.

Unfortunately, this bias is often a real problem. The real world simply doesn’t work the way search engines see it. Here’s an example: Suppose your busi- ness rents very expensive, specialized photographic equipment. Your busi- ness has the best prices and the best service of any company renting this equipment. Your local customers love you, and few other companies match your prices, service, or product range. So you decide to build a Web site to reach customers elsewhere, and ship rentals by UPS and FedEx.

The search engines base your rank, to a great degree, on the number and type of keywords in your pages.

To rank well, a competitor has added a bunch of pages about photography and photographic equipment to its site. To compete, you have to do the same. Do your customers care? No, they just want to find a particular piece of equipment that fills their need, rent it, and move on quickly. All the addi- tional information, the content that you’ve added, is irrelevant to them. It’s simply clutter.

This is a common scenario. I recently discussed the content issue with a client who was setting up a Web site at which people could quickly get a moving-service quote. The client wanted to build a clean, sparse site that allowed customers to get the quote within a couple of minutes. “But we don’t want all that stuff, that extra text, and nor do our clients!” he told me, and he had a good point.

You can’t ignore the fact that search engines like content. However, you can compete other ways. One of the most important ways is getting links from other sites, as you discover in blog 14. Search engines like to see links on other sites pointing to your site. Sites that have hundreds or thou- sands of other sites linking to them often rank well. But they still need at least some content for the search engines to index. And the best situation is lots of useful content with lots of incoming links.

Making Your Site Work Well

I’ve been writing about site design for almost seven years, and I’m happy to say that many of the rules of good site design just happen to match what search engines like. And many of the cool tricks that designers love cause problems with the search engines. So I want to quickly review a few tips for good site design that will help both your site visitors and the search engines work with your site.

Limiting multimedia
Most multimedia used on the Web is pointless because it rarely serves a useful purpose to the visitor. It’s there because Web designers enjoy working with it and because many people are still stuck in the old “you’ve got to be cool” mindset.

Look at the world’s most successful Web sites, and you’ll find that they rarely use multimedia — Flash animations and video, for example — for purely dec- orative purposes. Look at Amazon: Its design is simple, clean, black text on white background, with lots of text and very little in the way of animations, video, or sound (except, for instance, where it provides music samples in the site’s CD area). Look at Yahoo!, Google, CNN, or eBay — they’re not cool; they just get the job done.

You can employ multimedia on a Web site in some useful ways. I think it makes a lot of sense to use Flash, for instance, to create demos and presenta- tions. However, Flash intros are almost always pointless, and search engines don’t like them because Flash intros don’t provide indexable content. Anytime you get the feeling it would be nice to have an animation, or when your Web designer says you should have some animation, slap yourself twice on the face and then ask yourself this: Who is going to benefit: the designer or the site visitor? If that doesn’t dissuade you, have someone else slap you.

Using text, not graphics
A surprising number of Web sites use graphics to place text onto pages. Take a look at the Web site shown in Figure 4-1. Although this page appears to have a lot of text, every word is in an image file. Web designers often employ this technique so all browsers can view their carefully chosen fonts. But search engines don’t read the text in the images they run across, so this page provides no text that can be indexed by the search engines. Although this page may contain lots of useful keywords (you find out all about keywords in blog 5), the search engines read nothing. From a usability perspective, the design is bad, too, because all those images take much longer to download than the equivalent text would take.

Avoiding the urge to be too clever

I advise people to stay one step behind in Web technology and try not to be too clever. From a usability standpoint, the problem is that not all browser types work the same; they have different bugs and handle technical tricks differently.

If you’re always working with the very latest Web-development technology, more of your visitors are likely to run into problems. Cool technology often confuses the search engines, too. As an SEO friend likes to say, “Google likes black text on a white background.” In other words search engines like simple. The more complicated your Web pages are, the harder it is for search engines to read and categorize them.

You must strike a compromise between employing all the latest Web-design technology and tools and ensuring the search engines can read your pages. From a search-engine perspective, in fact, one step behind probably isn’t enough!

Don’t be cute

Some sites do everything they can to be cute. The Coca Cola site was a clas- sic example of this a few years ago, though it finally got the message and changed. The site had icons labeled Tour de Jour, Mind Candy, Curvy Canvas, Netalogue, and so on. What do these things mean? Who knows? Certainly not the site visitor.

This sort of deranged Web design is far less common now than it used to be, but you still see it occasionally — particularly in sites designed by hip Web- design firms. One incredibly irritating technique is the hidden navigation structure. The main page contains a large image with hotspots on it. But it’s unclear where the hotspots are, or what they link to, until you point at the image and move the mouse around. This strikes me as the Web-design equiv- alent of removing the numbers from the front of the homes in a neighbor- hood. You can still figure out where people live; you just have to knock on doors and ask.

Sweet and sickly cuteness doesn’t help your site visitors find their way around and almost certainly hurts you with the search engines.

Avoiding frames

Framed Web sites were very popular a few years ago; fortunately, they’ve fallen out of favor to a great degree. From a usability standpoint, there’s noth- ing wrong with frames if they’re used properly.

Here are a few reasons they’re less prevalent today:

Many designers misused frames, making sites hard to navigate. Too many frames in a browser window ignore the average Joe working with a small screen and low resolution.

Some browsers simply don’t handle frames well.

Search engines don’t handle frames well, for a whole list of reasons explained in blog 7.

I can think of few situations in which you can’t use some other mechanism rather than frames, so I advise you to stay away from them.

Making it easy to move around

Web design is constantly getting better, but it still surprises me that design- ers sometimes make it difficult for visitors to move around a Web site.

Think carefully about how your site is structured:

Does it make sense from a visitor’s standpoint? Can visitors find what they need quickly?

Do you have dangling pages — pages where a visitor can’t find a link to get back into your main site?

Search engines don’t like dangling pages, and consider what happens if someone on another site links directly to the page: Visitors can get to the page but not to the rest of your site.

Providing different routes

People think differently from each other, so you need to provide them with numerous avenues for finding their way around your site. And by doing so, you’re also giving more information to search engines and ensuring that search engines can navigate your site easily.

Here are some different navigational systems you can add to your site:

Sitemap. This page links to the different areas of your site, or even, in the case of small sites, to every page in the site. An example is www. peterkentconsulting.com/sitemap.htm .

Table of Contents or Index page. You can sort the page thematically or alphabetically.

Navigation bars. Most sites have navigation bars these days.

Navigation text links. Little links at the bottom of your pages, or along the sides, can help people find their way around . . . and the search engines, too.

I like to add simple text links near the top, rather than the bottom, of the page. Users with slow connections see these links quickly, and search engines are sure to find them. (Sometimes, on large and complex Web pages, search engines may miss links at the bottom of the page.)

Using long link text

It’s a proven fact that Web users like long link text — links that are more than just a single word and actually describe where the link takes you. Usability testing shows that long link text makes it much easier for visitors to find their way around a site. It’s not surprising if you think about it; a long link provides more information to visitors about where a link will take them.

Unfortunately, many designers feel constrained by design considerations, forcing all navigation links, for instance, to conform to a particular size. You often see buttons that have only enough room for ten or so characters, forc- ing the designer to think about how to say complicated things in one or two words.

Long links that explain what the referenced page is about are a great thing not only for visitors but also for search engines. By using keywords in the links, you’re telling the search engines what the referenced pages are about.

You also have a problem if all the links on your site are on image buttons — search engines can’t read images, so image buttons provide no information about the referenced page . You can’t beat a well-keyworded text link for pass- ing information about the target page to the search engines.

Don’t keep restructuring

Try to make sure your site design is good before you get too far into the process. Sites that are constantly being restructured have numerous problems, including the following:

Links from other Web sites into yours get broken, which is bad for potential visitors as well as for search engines (or, more precisely, bad for your position in the search engines because they won’t be able to reach your site through the broken links).

Anyone who has bookmarked your page now has a broken bookmark.

It’s a good idea to create a custom 404 error page, which is displayed in your browser if the server is unable to find a page you’ve requested. (Ask your Web server administrator how to do this; the process varies among servers.) Create an error page with links to other areas of the site, perhaps even a sitemap, so that if visitors and search-bots can’t find the right page, at least they’ll be able to reach some page on your site.

Editing and checking spelling

Check your pages for spilling and editing errors. Not only do error-free pages make your site appear more professional, they ensure that your valuable key- words are not wasted. If potential visitors are searching for rodent racing, for example, you don’t want the term rodent racing in your Web pages. (Except, that is, if you are trying to catch traffic from oft-misspelled keywords, some- thing I discuss in blog 5.)

Sometimes Web developers switch the attributes in the tag, putting the CON- TENT= first and then the NAME= , like this:

<META CONTENT=” your description goes here ”

NAME=” description ”>

Make sure that your tags do not switch the tag attributes. I don’t know if the order of the attributes causes a problem for Google or the other big search engines, but it does confuse some smaller systems. There’s no reason to do it, so don’t.

Giving search engines something to read

You don’t necessarily have to pick through the HTML code for your Web page to evaluate how search-engine-friendly it is. You can find out a lot just by looking at the Web page in the browser. Determine whether you have any text on the page. Page content — text that the search engines can read — is essen- tial, but many Web sites don’t have any page content on the front page and often have little or any on interior pages.

Here are some potential problems:
Having a (usually pointless) Flash intro on your site

Embedding much of the text on your site into images, rather than relying on readable text

Banking on flashy visuals to hide the fact that your site is light on content

Using the wrong keywords; blog 5 explains how to pick keywords

If you have these types of problems, they can often be time consuming to fix. (Sorry, you may run over the one-hour timetable by several weeks.) The next several sections detail ways you might overcome the problems.

Eliminating Flash

Huh? What’s Flash? You’ve seen those silly animations when you arrive at a Web site, with a little Skip Intro link hidden away in the page. Words and pic- tures appear and disappear, scroll across the pages, and so on. You create these animations with a product called Macromedia Flash.

I suggest that you kill the Flash intro on your site. I have very rarely seen a Flash intro that actually served any purpose. In most cases, they are nothing but an irritation to site visitors. (The majority of Flash intros are created because the Web designer likes playing with Flash.)

Replacing images with real text

If you have an image-heavy Web site, in which all or most of the text is embedded onto images, you need to get rid of the images and replace them with real text. If the search engine can’t read the text, it can’t index it.

It may not be immediately clear whether text on the page is real text or images. You can quickly figure it out a couple of ways:

Try to select the text in the browser with your mouse. If it’s real text, you can select it character by character. If it’s not real text, you simply can’t select it — you’ll probably end up selecting an image.

Use your browser’s View ➪ Source command to look at the HTML for the

page and then see if you can find the actual words in the text.


Using more keywords

The light-content issue can be a real problem. Some sites are designed to be light on content, and sometimes this approach is perfectly valid in terms of design and usability. However, search engines have a bias for content, for text they can read. (I discuss this issue in more depth in blog 9.) In general, the more text — with the right keywords — the better.

Using the right keywords in the right places

Suppose that you do have text, and plenty of it. But does the text have the right keywords? The ones uncovered at Yahoo! Search Marketing earlier in this blog? It should.

If the reference to Yahoo! Search Marketing doesn’t ring a bell, check out the “Picking Good Keywords” section, earlier in this blog.

Where keywords are placed and what they look like are also important. Search engines use position and format as clues to importance. Here are a few simple techniques you can use — but don’t overdo it!

Use particularly important keywords — those that people are using to search for your products and services — near the top of the page. Place keywords into <H> (heading) tags.

Use bold and italic keywords; search engines take note of this.
Put keywords into bulleted lists; search engines also take note of this. Use keywords multiple times on a page, but don’t use a keyword or key- word phrase too often. If a word makes up more than, say, 8 to 10 per- cent of all the words on the page, it may be too much.

Make sure that the links between pages within your site contain keywords. Think about all the sites you’ve visited recently. How many use links with no keywords in them? They use buttons, graphic navigation bars, short little links that you have to guess at, click here links, and so on. Big mistakes.

I don’t object to using the words click here in links. Some writers have sug- gested that you should never use click here because it sounds silly and because people know they’re supposed to click. I disagree, and research shows that using the words can sometimes increase the number of clicks on a link. The bottom line is that you should rarely, if ever, use a link with only the words click here in the link text; you should include keywords in the link.

When you create links, include keywords in the links wherever possible. If on your rodent-racing site you’re pointing to the scores page, don’t create a link that says To find the most recent rodent racing scores, click here or, perhaps, To find the most recent racing scores, go to the scores page. Instead, get a few more keywords into the links, like this: To find the most recent racing scores, go to the rodent racing scores page.

Getting Your Site Indexed

So your pages are ready, but you still have the indexing problem. Your pages are, to put in bluntly, just not in the search engine! How do you fix that problem?

For Yahoo! Directory and the Open Directory Project, you have to go to those sites and register directly, but before doing that, you should read blog 12. With Google, Yahoo! Web Search, MSN, and Ask.com, the process is a little more time consuming and complicated.

The best way to get into the search engines is to have them find the pages by following links pointing to the site. In some cases, you can ask or pay the search engines to come to your site and pick up your pages, but you face two main problems with this:

If you ask search engines to index your site, they probably won’t do it. And if they do come and index your site, it may take weeks or months. Asking them to come to your site is unreliable.

If you pay search engines to index your site, you have to pay for every URL you submit. The problem with paying, of course, is that you have to pay.

If you want to submit your site to the search engines for indexing, read blog 11, where I provide all the details.

So how do you get indexed? The good news is that you can often get indexed by some of the search engines very quickly. I’m not talking about a full-blown link campaign here, with all the advantages I describe in blogs 14 and 15. You simply want to get search engines — particularly Google, Yahoo!, MSN, and Ask.com — to pick up the site and index it.

Find another Web site to link to your site, right away. Call several friends, col- leagues, and relatives who own or control a Web site, and ask them to link to your site. Of course, you want sites that are already indexed by the search engines. The searchbots have to follow the links to your site.


 When you ask friends, colleagues, and relatives to link to you, specify what you want the links to say. No click here or company name links for you. You want to place keywords into the link text. Something like Visit this site for all your r odent    racing needs - mice, rats, stoats, gerbils, and all other kinds of r odent    racing . Keywords in links are a powerful way to tell a search engine what your site is about.

After the sites have links pointing to yours, it can take from a few days to a few weeks to get into the search engines. With Google, if you place the links right before Googlebot indexes one of the sites, you may be in the index in a few days. I once placed some pages on a client’s Web site on a Tuesday and found them in Google on Friday. But Google can also take several weeks to index a site. The best way to increase your chances of getting into the search engines quickly is to get as many links as you can on as many sites as possible.


Planning Your Search-Engine Strategy

Avoiding problems with your Web designer Evaluating the competition Understanding the search tail The six search engine variables Planning your attack

here’s lot to discover about generating traffic from the search engines, and sometimes it’s hard to see the forest for the trees. As you discover in this blog, there’s page optimization and link strategies and index submissions and directory submissions and electronic press releases and blogs and this and that . . . it goes on and on. Before you jump right in, I need to discuss the big picture, to give you an idea of how all this fits together and help you decide what you should do when . . . to help you plan your strategy. In this blog I show you how a search-engine campaign works overall.


Don’t Trust Your Web Designer

Let me start with a warning: Don’t rely on your Web designer to manage your SEO project. In fact, I know that many of you are reading this blog because you did just that, and have realized the error of your ways.

Last week I consulted with the owner of a small e-commerce store. He’d just paid a Web-design firm $5,000 to build his site, and before beginning he had asked them to make sure the site was “search-engine friendly.” Unfortunately, that means different things to different people, and to the design firm it didn’t mean much. The site they built definitely was not optimized for the search engines. The owner asked the firm what it was planning to do about the search engines. They told him it would cost him $5,000.

This unusual case is more egregious than most, but the first part — that your Web-design firm says it will handle the search engines, then doesn’t — is very common. When I hire a Web designer to build a site for me, I explain exactly what I want. And you should do the same. (Thus, this blog can help you even if you never write a line of HTML code.)

The problem is twofold:

Web designers pretty much have to say they understand the search engines, because all of their competitors are saying it.

Many Web designers think they do understand, but typically it’s at a “add some meta tags and submit to the search engines” level. It won’t work.

Sorry, Web designers. I don’t want to be rude, but this is a simple fact, attested to by many, many site owners out there. I’ve seen it over and over again. Not trusting your Web designer or team is probably the first step in your search-engine strategy!

Understanding the Limitations

You’ve probably received spam e-mails guaranteeing top-ten positions for your Web site in the search engines. You’ve probably also seen claims that you’ll be ranked in hundreds or thousands of search engines. Most of this is nonsense — background noise that creates an entirely false picture. As one of my clients put it, “There’s a lot of snake oil out there!” Here are the facts.

Typically, getting a high position isn’t that easy. You try a couple of tech- niques, but they don’t seem to work. So you try something else, and maybe you achieve a little success. Then you try another thing. Search engine opti- mization can often be very labor intensive, and you may not see results for weeks, and more likely, months.

Big doesn’t always equal better

By the way, don’t imagine that if you’re working with a large Web-design team with extensive programming experience they understand the search engines either. In fact, it’s sometimes the more sophisticated design teams that get into the most trouble, building complex sites that simply won’t work well with the search engines.
 I consult with companies big and small, so I’ve advised large design teams made up of very good programmers. I can assure you that large, sophisticated teams often know as little as the independent Web designer who’s been in busi- ness a few months.


Top two in four

Sometimes it’s easy to get a very high position in the search systems. But usually it isn’t. A client wanted to be positioned in Google for six impor- tant key phrases. I built some pages, ensured that Google knew where those pages were (find
 out how to do this in blog 12), and waited. In just four days, the client didn’t just have a top- ten position or even just a number-one position, but the top two positions for five of the six key phrases. But this situation is very unusual.

The degree of work required depends on the competitiveness of the keywords you are going after. Some keywords are incredibly competitive: mortgage, insur- ance, attorney, real estate , and so on, are highly competitive, with millions of people wanting some of the action. Other phrases are very easy — phrases such as rodent racing , for instance. If you’re in the rodent-racing business, you’re in luck, because you can probably rank right at the top very easily!

Although the way that search engines function is based on science, search engine optimization is more art than science. Why? Because the search engines don’t want outside parties to know exactly how they rank sites. You have to just experiment. Ranking a site can be very difficult, and tremen- dously laborious. After all, why should it be easy? There is huge competition, so it can’t always be easy. If it were easy for your site, then it would be easy for your competitors’ sites, wouldn’t it? And, after all, there can only ever be one number one.

Eyeing the Competition
Some search terms are incredibly competitive. That is, many, many sites are competing for the top positions. Other search terms are far less competitive. How can you tell just how competitive your search terms are? Let me show you a few ways to figure it out:

Search for your terms. This is not a terribly good method, but so com- monly recommended I want to explain it. Go to Google and search for a few of your terms. (I discuss keywords in more detail in blog 5.) For instance, search for personal injury lawyer . You see a blue bar containing something like this:

Results 1 - 10 of about 16,300,000 for personal injury lawyer

This tells you that over 16M pages match the search terms in the Google index. Actually, most of these pages don’t match well. Most of the pages don’t actually have the term personal injury lawyer. Rather, as explained earlier, they have the words personal, injury, and lawyer scattered around the page.

Search for your terms using quotation marks. Type search terms in quotation marks, like this: “personal injury lawyer.” This time Google searches for the exact phrase, and comes back with a different number. When I searched, it came back with 3,200,000, because Google ignores all the pages with the words scattered around the page, and returns only pages with the exact phrase.

Here’s the problem with these two techniques: While they show you how commonly used the words are, they don’t show you how well the pages are optimized. Remember, you’re not competing against every page with these terms; you’re really competing with pages that were “optimized” for the search engines. There may be millions of pages with the term, but if none of them have been optimized, you can take your new-found SEO knowledge, create your own optimized pages, and have a good chance of ranking well.

So here’s another quick technique I like to use — a simple way to get a feel for competitiveness in a few seconds. Search for a term, then scan down the page looking for the number of

PPC ads on the page. As you look down the page, you see three PPC ads at the top of the page, then more ads all the way down the right side of the page. Lots of PPC ads indicate lots of interest in the phrase. If people are spending money on PPC ads, many are also proba- bly spending money on SEO.

Bold words on the page. You’ll also notice that Google bolds the words that you searched for; all the major search sites do this. Lots of bold words often mean well-optimized pages.

Bold words in the links (page titles). Bold words in each page result’s link indicate that someone has been optimizing the pages; the links are the page titles. The more bold text you see as you scan down, the more competitive the search terms are likely to be.

Complete phrases on the page. The more frequently you see the full phrase you searched for, the more competitive the terms are likely to be; if the search engine returns mostly pages with the words scattered around, it’s not very competitive.

Here’s another example. Search Google for rodent racing. What do you see?. First, notice almost no PPC ads (and the ad that does appear is not well matched to the search term). Next, notice very little bold text on the page, and none in the page titles (the links at the top of each search result). The matches in this case are all full phrases; but still, the other three factors suggest that this is not a very competitive term. You can see the difference between these two pages. The first search term, personal injury lawyer, is far more competitive than the second, rodent racing .

Get your rodent running

Here’s an example of how uncompetitive the phrase rodent racing actually is. These results are skewed somewhat, because I used the term rodent racing in the first edition of this blog. Results 1, 2, 5, and 8 point to sample blogs
 Amazon.com blog page. (Ever wondered by Amazon has recently started dropping large numbers of keywords onto their product pages?) My blog has four entries out of the first ten, without even trying!

from the blog posted online, and to the

How important is competitiveness? When targeting search terms that are not very competitive, you may be able to create a few optimized pages and rank well. In very competitive areas, though, creating a few nicely optimized pages isn’t enough. You must have links pointing to the site (perhaps to many of them), and you may also need large numbers of pages. In some really compet- itive areas, it may take hundreds, if not thousands, of links.

By the way, I often have clients ask me why a competitor ranks so well. “Their pages aren’t better optimized than mine,” I often hear. “We have more pages, and more pages with the right keywords...why is his site ranked so well?”

Without understanding all the variables — which we discuss later in this blog, under “Controlling the Search-Engine Variables” — you can’t tell for sure why a site ranks well. Is it because the site has been around much longer than yours? Is it because it has more content? Because the content is better “optimized”? Because the site has more incoming links? In fact the last of these factors is often essential; when I do a link analysis (see blog 15) I often discover that a poorly optimized site has a huge number of incoming links, with just the right keywords.

Going Beyond Getting to #1

Everyone wants to rank #1 for the top keywords. Lawyers want to rank #1 for attorney or lawyer. Real estate agents want to rank #1 for real estate. Shoe stores want to rank #1 for shoes, and so on.

But what does being #1 achieve? You’re trying to get people to your Web site, not to get any particular position, right? Getting ranked in the search engines is merely a way to generate that traffic to your site. People often assume that to generate traffic, they have to get #1 positions for top keywords. That’s not the case. You can generate plenty of traffic to your site without ever getting to #1 for the most popular phrases. And in many cases, the traffic arriving at your site will be better — the visitors will be more appropriate for your site. There are two things to understand: highly targeted keyword phrases , and the search tail .

Highly targeted keyword phrases

If your keywords are very competitive, look for keywords that aren’t so sought after:

Go local. One common strategy is, of course, to focus on local key- words. If you’re a real estate agent, don’t target real estate . Instead, target real estate in your area: denver realtor, chicago real estate, dallas homes for sale, and so on.

Focus on more specialized search terms. A realtor might target traffic on keywords related to commercial real estate, or condos, for instance. Incorporate spelling mistakes. Some realtors target the very common misspelling realtor.

These specialized search terms are hidden away in the search “tail,” so now let’s look at that concept.

Understanding the search tail

The search tail is an important concept to understand. While the first few top keywords may get far more searches than any other search, when you look at the total number of searches, the top terms actually account for only a small percentage of the searches.

 Wordtracker, is a great little tool that shows what search terms people are typing into the search engines. I searched for video games and Wordtracker returned 300 results containing that term. I don’t have room for 300, so I’ve shown the first few.

video games

music video games

adult video games

used video games

video games xbox

video games playstation 2

violent video games

online video games

sex video games

free video games

history of video games

xxx video games

video games game cube

trade video games

violence in video games

cheap video games

nude video games

video poker games

Page optimization

Content is just a start. Content has to be placed onto the pages in the correct way; the pages must be optimized to get the most out of the keywords. As you read in blogs 2 and 6, you must place the words onto the pages in the correct places and formats.

If a search engine finds the relevant keywords on your page, that’s good. If it finds the keywords in the right places on the page, that’s a really powerful thing that differentiates your page from your competitors’.

Submissions

In some ways submissions — submitting information to the search engines telling them where your pages can be found and asking them, in effect, to come to your site and index it — is not as important as many people imagine. Many businesses have, for a long time, promoted the idea that you have to submit your pages to the search engines, when in fact up until mid-2005 it really didn’t matter much. You could submit, but the search engines would quite likely ignore the submission; links are what really counted.

However, in 2005 Google, introduced a new concept, the sitemap , and was quickly followed by Yahoo! . This file is placed into your Web site’s root direc- tory containing a list of links to all your pages, so the search engines could more easily find them.

These days I recommend you assume that submitting to the search engines will not get you indexed — that the way to get indexed is by making sure the search engines find links to your site — but that you also Provide Google and Yahoo! sitemaps, so those search engines can use them if they decide to (they may not). You read more about this in blog 11.

Links

Links pointing to your Web site are incredibly important in a competitive key- word market. If you’re targeting rodent racing, you probably don’t need to worry too much about links (though every site needs at least some incoming links — links pointing to your site from other sites). But if you have lots of competition vying for your keywords, you won’t win without links.

Links are so important, in fact, that a page can rank in the first position in any of the three major search engines, even if the page does not have the keywords that have been searched for . . . as long as links pointing to the page have the keywords. I explain this in blog 14. The more competitive your area, the more important links become.

Google

I’ll start with the behemoth: Google. Here’s the quickest and easiest way to see what Google has in its index. Search Google, either at the site or through the Google toolbar (see blog 1) for this:

site: domain .com

Don’t type the www. piece, just the domain name. For instance, say your site’s domain name is RodentRacing.com . You’d search for this:

site:rodentracing.com

Google returns a list of pages it’s found on your site; at the top, on the blue bar, you see something like this:

Results 1 - 10 of about 256 from rodentracing.com

That’s it — quick and easy. You know how many pages Google has indexed on your site, and can even see which pages.

Here’s another way to see what’s in the index, in this case a particular page in your site. Open your browser and load a page at your site. Then follow these steps:

1. Click the i icon on the Google toolbar.

I’m assuming that you’re using Internet Explorer and have, as I suggest in blog 1, downloaded the Google toolbar — available at toolbar. google.com — to your computer. If you don’t have the toolbar, don’t worry; I explain a non-toolbar method in a moment.

2. Select Cached Snapshot of Page from the drop-down list that appears. If you’re lucky, Google loads a page showing you what it has in its cache, so you know Google has indexed the page. (See Figure 2-1.) If you’re unlucky, Google tells you that it has nothing in the cache for that page. That doesn’t necessarily mean Google hasn’t indexed the page, though.

A cache is a temporary storage area in which a copy of something is placed. In the context of the Web, a cache stores a Web page. Google, Yahoo!, MSN, and Ask.com keep a copy of many of the pages they index, and all but Yahoo! even tell you the date that they indexed the cached pages.

If you don’t have the Google Toolbar, you can instead go to Google ( www. google.com ) and type the following into the Google search box:

cache:http:// yourdomain.com / page.htm

A page stored in the Google cache.

Your One-Hour Search-Engine-Friendly Web Site Makeover

Replace yourdomain.com with your actual domain name, and page.htm with the actual page name, of course. When you click Search, Google checks to see if it has the page in its cache.

What if Google doesn’t have the page? Does that mean your page isn’t in Google? No, not necessarily. Google may not have gotten around to caching it. Sometimes Google grabs a little information from a page but not the entire page.

By the way, you can see a cached page saved by Google, Yahoo!, or MSN directly from the search results; look for the Cached or Cached page link after a search result.

Yahoo! and MSN

And now, here’s a bonus. The search syntax I used to see what Google had in its index for rodentracing.com — site:rodentracing.com — not only works on Google, but also on Yahoo! and MSN. That’s right, type the same thing into any of these search sites and you see how many pages on the Web site are in the index . . . with one caveat. MSN, at the time of writing at least, is a little flaky.

MSN reports a much higher number when you first run this search; but view a later page, and this number drops. For instance, MSN may show you this:

Page 1 of 456 results containing

site:peterkentconsulting.com

At the bottom of the page you see the page-navigation numbers, like this:

1 2 3 4 5 Next

I suggest you click the number 5, to move to Page 5 in the results. Then you see this:

Previous 1 2 3 4 5 6 7 8 9 Next

Click page 9, and continue this way until you get to the last page or until the Page x of xxx line at the top changes. You then see the actual number of pages that MSN has in its index. Why is this? Just a bug; by the time you read this, in fact, it may not be necessary to do this, MSN may have fixed the prob- lem, but it’s been there for some time now.

You can search for a Web site at Google another way, too. Simply type the domain name into the Google search box and click Search. Google returns just that site’s home page. If you want to use the search box on the Google toolbar to do this, type the domain name and then click the binoculars button. (If you type the domain name and press Enter, Google simply redi- rects your browser to the specified domain name.)

Yahoo! Directory

You must check whether your site is listed in the Yahoo! Directory. You have to pay to get a commercial site into the Yahoo! Directory, so you may already know if you’re listed there. Perhaps you work in a large company and suspect that another employee may have registered the site with Yahoo! Here’s how to find out:

1. Point your browser to dir.yahoo.com .

This takes you directly to the Yahoo! Directory search page. 2. Type your site’s domain name into the Search text box.

All you need is yourdomain.com , not http://www. or anything else. 3. Make sure that the Directory option button is selected, then click

Search.
If your site is in the Yahoo! Directory, your site’s information appears on the results page. You may see several pages, one for each category in which the site has been placed (though in most cases a site is placed into only one category).


Open Directory Project

You must also know if your site is listed in the Open Directory Project ( www. dmoz.org ). If it isn’t, it should be. Just type the domain name, without the www. piece. If your site is in the index, the Open Directory Project will tell you. If it isn’t, you’d better register it; see blog 12.

Taking Action if You’re Not Listed

First, if your site isn’t in Yahoo! Directory or the Open Directory Project, you have to go to those systems and register your site. See blog 12 for infor- mation. What if you search for your site in the search engines and can’t find it? If the site isn’t in Google, Yahoo!, and MSN, you have a huge problem.

Here are two possible reasons your site isn’t being indexed in the search engines:

The search engines haven’t found your site yet. The solution is relatively easy, though you won’t get it done in an hour.

The search engines have found your site, but can’t index it. This is a serious problem, though in some cases you can fix it quickly.

For the lowdown on getting your pages indexed in the search engines — to ensure that the search engines can find your site — see the section “Getting Your Site Indexed,” later in this blog. To find out how to make your pages search-engine-friendly — to ensure that once found, your site will be indexed well — check out the section “Examining Your Pages,” later in this blog. But first, let’s see how to check to see if your site can be indexed.

Is your site invisible?

Some Web sites are virtually invisible. A search engine might be able to find the site (by following a link, for instance). But when it gets to the site, can’t read it or, perhaps, can read only parts of it. A client (before he was a client) built a Web site that had only three visible pages; all the other pages, includ- ing those with product information, were invisible.

How does a Web site become invisible? I talk about this subject in more detail in blog 7, but here’s a brief explanation:

The site is using some kind of navigation structure that the search engines can’t read, so they can’t find their way through the site.

The site is creating dynamic pages that the search engines choose not to read.

Unreadable navigation

Many sites have perfectly readable pages, with the exception that the search- bots — the programs the search engines use to index Web sites — can’t nego- tiate the site navigation. The searchbots can reach the home page, index it, and read it, but they can’t go any further. If, when you search Google for your pages, you find only the home page, this is likely the problem.

Why can’t the searchbots find their way through? The navigation system may have been created using JavaScript, and because search engines ignore JavaScript, they don’t find the links in the script. Look at this example:

<SCRIPT TYPE=”javascript” SRC=”/menu/menu.js”></SCRIPT>

In one site I reviewed, this was how the navigation bar was placed into each page: The page called an external JavaScript, held in menu.js in the menu subdirectory. The search engines won’t read menu.js , so they’ll never read the links in the script.

Try these simple ways to help search engines find their way around your site, whether or not your navigation structure is hidden:

Create more text links throughout the site. Many Web sites have a main navigation structure and then duplicate the structure by using simple text links at the bottom of the page. You should do the same.

Add a sitemap page to your site. This page contains links to most or all of the pages on your Web site. Of course, you also want to link to the sitemap page from those little links at the bottom of the home page.

Dealing with dynamic pages

In many cases, the problem is that the site is dynamic — that is, a page is cre- ated on the fly when a browser requests it. The data is pulled out of a data- base, pasted into a Web page template, and sent to the user’s browser. Search engines often won’t read such pages, for a variety of reasons explained in detail in blog 7.

How can you tell if this is a problem? Take a look at the URL in the browser’s location bar. Suppose that you see something like this:

http://www.yourdomain.edu/rodent-racing-scores/march/index.php

This address is okay. It’s a simple URL path made up of a domain name, two directory names, and a filename. Now look at this one:

http://www.yourdomain.edu/rodent-racing/scores.php?prg=1

The filename ends with ?prg=1 . This parameter is being sent to the server to let it know what information is needed for the Web page. If you have URLs like this, with just a single parameter, they’re probably okay, especially for Google; however, a few smaller search engines may not like them. Here’s another example:

http://yourdomain.com/products/index.html?&DID=18&CATID=13

&ObjectGroup_ID=79

This one may be a real problem, depending on the search engine. This URL has too much weird stuff after the filename:

?&DID=18&CATID=13&ObjectGroup_ID=79 . That’s three parameters — DID=18 , CATID=13 , and ObjectGroup_ID=79 — which are too many. Some systems cannot or will not index this page. (My feeling is that Google tends to index “deeper” into dynamic sites than, for instance, Yahoo!)

Another problem is caused by session IDs — URLs that are different every time the page is displayed. Look at this example:

http://yourdomain.com/buyAHome.do;jsessionid=07D3CCD4D9A6A

9F3CF9CAD4F9A728F44

Each time someone visits this site, the server assigns a special ID number to the visitor. That means the URL is never the same, so Google won’t index it.

Search engines may choose not to index pages with session IDs. If the search engine sees links to a page that appears to have a session ID, it doesn’t know whether the URL will change between sessions or whether many different URLs point to the same page. Search engines don’t want to overload the site’s server and don’t want garbage in their indexes.

If you have a clean URL with no parameters, the search engines should be able to get to it. If you have a single parameter in the URL, it’s probably fine. Two parameters may not be a problem, although they’re more likely to be a problem than a single parameter. Three parameters are almost certainly a problem with some search engines. If you think you have a problem, I suggest reading blog 7.


Picking Good Keywords

Getting search engines to recognize and index your Web site can be a prob- lem, as the first part of this blog makes clear. Another huge problem — one that has little or nothing to do with the technological limitations of search engines — is that many companies have no idea what keywords (the words people are using to search for Web sites at the search engines) they should be using. They try to guess the appropriate keywords, without know- ing what people are really using in the search engines.

In blog 5, I explain keywords in detail, but here’s how to do a quick key- word analysis:

1. Point your browser to http://searchmarketing.yahoo.com/rc/srch/ .

You see the Yahoo! Search Marketing Resource Center; Search Marketing is Yahoo!’s PPC (pay-per-click) division. See blog 17 for more about PPC.

2. Click the Advertiser Keyword Selector Tool link on the right side of the page.

A small window opens with a search box.

3. In the search box, type a keyword you think people may use to search

for your products or services.

4. Press Enter.

The tool returns a list of keywords, showing you how often that term and related terms are used by people searching on Yahoo! and partner sites. See Figure 2-2.

I’m tired of looking for the Yahoo! keyword tool, and having to explain to people how to find it. It keeps moving! So I’ve placed a link on my site; go to http://searchenginebulletin.com/yahoo-keywords.html .

You may find that the keyword you guessed is perfect. Or you may discover better words, or, even if your guess was good, find several other great key- words. A detailed keyword analysis almost always turns up keywords or keyword phrases you need to know about.

Don’t spend a lot of time on this task. See if you can come up with some useful keywords in a few minutes and then move on; see blog 5 for details about this process.

Examining Your Pages

Making your Web pages “search-engine-friendly” was probably not uppermost in your mind when you sat down to design your Web site. That means your Web pages — and the Web pages of millions of others — probably have a few problems in the search-engine-friendly category. Fortunately, such problems are pretty easy to spot; you can fix some of them quickly, but others are more troublesome.

Using frames

In order to examine your pages for problems, you need to read the pages’ source code; remember, I said you’d need to be able to understand HTML!

In order to see the source code, choose View ➪ Source in your browser.

When you first peek at the source code for your site, you may discover that your site is using frames. (Of course, if you built the site yourself, you already know whether it uses frames. However, you may be examining a site built by someone else.) You may see something like this in the page:

<HTML> <HEAD>

</HEAD>

<FRAMESET ROWS=”20%,80%”> <FRAME SRC=”navbar.html”> <FRAME SRC=”content.html”>

</FRAMESET>

<BODY>

</BODY> </HTML>

When you choose View ➪ Source in Internet Explorer, you’re viewing the

source of the frame-definition document, which tells the browser how to set up the frames. In the preceding example, the browser creates two frame rows, one taking up the top 20 percent of the browser and the other taking up the bottom 80 percent. In the top frame, the browser places content taken from the navbar.html file; content from content.html goes into the bottom frame.

Framed sites don’t index well. The pages in the internal frames get orphaned in the search engines; each page ends up in search results alone, without the navigation frames with which they were intended to be displayed.

Framed sites are bad news for many reasons. I discuss frames in more detail in blog 7, but here are a few quick fixes:

Add TITLE and DESCRIPTION tags between the <HEAD> and </HEAD> tags. (To see what these tags are and how they can help with your frame issues, check out the next two sections.)

Add <NOFRAMES> and </NOFRAMES> tags between the <BODY> and </BODY> tags, and place 200 to 300 words of keyword-rich content between the tags. The NOFRAMES text is designed to be displayed by browsers that can’t work with frames, and search engines will read this text, although they won’t rate it as high as normal text (because many designers have used NOFRAMES tags as a trick to get more keywords into a Web site).

Include a number of links, in the text between the NOFRAMES tags, to other pages in your site to help the search engines find their way through.

Looking at the TITLE tags

TITLE tags tell a browser what text to display in the browser’s title bar, and they’re very important to search engines. Quite reasonably, search engines figure that the TITLE tags may indicate the page’s title — and therefore its subject.

Open your site’s home page and then choose View ➪ Source (in Internet

Explorer) to view the page source. A text editor opens, showing you what the page’s HTML looks like. Here’s what you should see at the top of the page:

<HTML> <HEAD>

<TITLE> Your title text is here </TITLE>

Here are a few problems you may have with your TITLE tags:

They’re not there. Many pages simply don’t have TITLE tags. If not, you’re not giving the search engines one of the most important pieces of information about the page’s subject matter.

They’re in the wrong position. Sometimes you find the TITLE tags, but they’re way down in the page. If they’re too low in the page, search engines may not find them.

They’re there, but they’re poor. The TITLE tags don’t contain the proper keywords.

Your TITLE tags should be immediately below the <HEAD> tag and should contain useful keywords. Have around 40 to 60 characters between the <TITLE> and </TITLE> tags (including spaces) and, perhaps, repeat the pri- mary keywords once. If you’re working on your Rodent Racing Web site, for example, you might have something like this. Find out more about keywords in blog 5:

<TITLE>Rodent Racing Info. Rats, Mice, Gerbils, Stoats,

all kinds of Rodent Racing</TITLE>

Examining the DESCRIPTION tag

The DESCRIPTION tag is important because search engines often index it (under the reasonable assumption that the description describes the con- tents of the page) and, in some cases, may use the DESCRIPTION tag to pro- vide the site description on the search-results page.

In most cases these days, the major search engines usually don’t use the DESCRIPTION tag to provide the description in the search results. Instead, they typically find the search words in the page, grab a snippet of informa- tion from around the words, and use that as the description. (In some cases, Google and MSN may grab the description from the Open Directory project, while Yahoo! may use the description from the Yahoo! Directory.)


In some cases, though, if the search engine can’t find the keywords in the page (if it finds the page based on its TITLE tag, for example, or links point- ing at the page rather than page content), it may use the DESCRIPTION tag.

Open a Web page, then open the HTML “source” (select View ➪ Source from

your browser’s menu), and take a quick look at the DESCRIPTION tag. It should look something like this:

<META NAME=”description” CONTENT=”your description goes

here”>

Sites often have the same problems with DESCRIPTION tags as they do with TITLE tags. The tags aren’t there, are hidden away deep down in the page, or simply aren’t very good.

Place the DESCRIPTION tag immediately below the TITLE tags (see Figure 2-3) and create a keyworded description of up to 250 characters (again, including spaces). Here’s an example:

<META NAME=”description” CONTENT=”Rodent Racing - Scores,

Schedules, everything Rodent Racing. Whether you’re into mouse racing, stoat racing, rats, or gerbils, our site provides everything you’ll ever need to know about Rodent Racing and caring for your racers.”>

When you find a new search system, look carefully on the page near the search box, or on the search results page, and you may find where the search results are coming from. For instance, if you use Alexa ( www.Alexa.com ), one of Amazon.com’s search engines, you see the words POWERED BY GOOGLE next to the Search button; use Amazon.com’s other search system, A9 ( www.A9.com ), and at the bottom of the search results you see this:

Search results enhanced by Google. Results also provided

by a9.com and Alexa.

The same search run at all three systems — Alexa, A9, and Google — produces very similar results. (Although a site may get its search-results feed from another one place, it may manipulate the results so they’re listed in a slightly different order.)

You’ll also want to work with some other search systems, as you find out in articles 12 and 13. In some cases, you need to check out specialty directo- ries and indexes related to the industry in which your Web site operates. But the preceding systems are the important ones for every Web site.

Google alone provides well over 50 percent of all search results (down from 75 percent just a year or two ago). Get into all the systems on the preceding list, and you’re in front of probably more than 95 percent of all searchers. Well, perhaps you’re in front of them. You have a chance of being in front of them, anyway, if your site ranks highly (which is what this book is all about).

Search Engine Magic

Go to Google and search for the term personal injury lawyer. Then look the blue bar below the Google logo, and you see something like this:

Results 1 - 10 of about 16,300,000 for personal injury

lawyer

This means Google has found over 16M pages that contain these three words. Yet somehow it has managed to rank the pages. It’s decided that one particular page should appear first, then another, then another, and so on, all the way down to page 16,300,000. (By the way, this has to be one of the wonders of the modern world: The search engines have tens of thousands of computers, evaluating 10 or 20 billion pages, and returning the information in a fraction of a second.)


How do they do it?

How on earth does Google do it? How does it evaluate and compare pages? How do other search engines do the same? Well, I don’t know exactly . The search engines don’t want you to know how they work (or it would be too easy to create pages that exactly match the search system, “giving them what they want to see”). But I can explain the general concept.

When Google searches for your search term, it begins by looking for pages containing the exact phrase. Then it starts looking for pages containing the words close together. Then it looks for pages that have the pages scattered around. This isn’t necessarily the order in which a search engine shows you pages; in some cases, pages with words close together (but not the exact phrase) appear higher than pages with the exact phrase, for instance. That’s because search engines evaluate pages according to a variety of criteria.

The search engines look at many factors. They look for the words throughout the page, both in the visible page and in the HTML source code for the page. Each time they find the words, they are weighted in some way. A word in one position is “worth” more than a word in another position. A word formatted in one way is “worth” more than a word formatted in another.

There’s more, though. The search engines also look at links pointing to pages, and uses those links to evaluate the referenced pages: How many links are there? How many are from popular sites? What words are in the link text?

Stepping into the programmers’ shoes

There’s a lot of conflicting information out there about SEO. Some of it’s good, some of it’s not so good, and some of it’s downright wrong. When evaluating a claim about what the search engines do, I sometimes find it useful to step into the shoes of the people building the search engines; I try to think, “what would make sense” from the perspective of the programmers who write the code that evaluates all these pages.

Consider this: Say you search for personal injury lawyer , and the search engine finds one page with the term in the pages title” (between the <title> and </title> tags), and another page with the term somewhere deep in the page text. Which do you think is likely to match the search term better? If the text is in the title, doesn’t that indicate that page is likely to be, in some way, related to the term? If the text is deep in the body of the page, couldn’t it mean that the page isn’t directly related to the term, that it is related to it in some incidental or peripheral manner?

Considering SEO from this point of view makes it easier to understand how the search engines try to evaluate and compare pages. If the keywords are in the links pointing to the page, the page must be relevant to those keywords; if the keywords are in headings on the page, that must be significant; if the key- words appear frequently throughout the page, rather than just once, that must mean something. All of sudden, it all makes sense.

 I discuss things that the search engines don’t like. You may hear elsewhere all sorts of warnings that may or may not be correct. Here’s an example: I’ve read that using a refresh meta tag to automatically push a visitor from one page to another will get your site penalized, and may even get your site banned from the search engine. You’ve seen this situation: You land on a page on a Web site, and there’s a message saying something like “We’ll forward you to page x in five seconds, or you can click here.” The theory is that search engines don’t like this, and they may punish you for doing this.

Now, does this make any sense? Aren’t there good reasons to sometimes use such forwarding techniques? Yes, there are. So why would a search engine punish you for doing it? They don’t. They probably won’t index the page that is forwarding a visitor — based on the quite reasonable theory that if the site doesn’t want the visitor to read the page, they don’t need to index it — but you’re not going to get punished for using it.

Remember that the search-engine programmers are not interested in punish- ing anyone, they’re just trying to make the best choices between billions of pages. In general, search engines use their “algorithms” to determine how to rank a page, and try to adjust the algorithms to make sure “tricks” are ignored. But they don’t want to punish anyone for doing something for which there might be a good reason, even if the technique could also be used as a trick.

I like to use this as my “plausibility filter” when I hear someone make some unusual or even outlandish claim about how the search engines function. What would the programmers do? , I ask myself.

Gathering Your Tools

You need several tools and skills to optimize and rank your Web site. I talk about a number of these in the appropriate articles, but I want to cover a few basics before I move on. It goes without saying that you need

Basic Internet knowledge.

A computer connected to the Internet. A Web site.

One of these two things:

• Good working knowledge of HTML.

• Access to a geek with a good working knowledge of HTML. Which path should you take? If you don’t know what HTML means (HyperText Markup Language), you probably need to run out and find that geek. HTML is the code used to create Web pages, and you need to understand how to use it to optimize pages. Discussing HTML and how to upload pages to a Web site is beyond the scope of this book. If you’re interested in finding out more, check out HTML For Dummies, 4th Edition, by Ed Tittel and Natanya Pitts, and Creating Web Pages For Dummies, 6th Edition, by Bud Smith and Arthur Bebak (both published by Wiley).

Toolbars. Install the Google toolbar in your Web browser . . . and per- haps the Yahoo!, MSN, and Ask.com toolbars, too. And, maybe, the Alexa toolbar. (Before you complain about spyware, I explain in a few moments.) You may want to use these tools even if you plan to use a geek to work on your site. They’re simple to install and open up a com- pletely new view of the Web. The next two sections spell out the details.

Search toolbars

I definitely recommend the Google toolbar, which allows you to begin search- ing Google without going to the Web site first. In addition, you might want to use the Yahoo! and MSN toolbars, which do the same. (You might also try the Ask.com toolbar, but remember that Ask is far less important than Google, Yahoo!, and MSN.) In addition, these toolbars have plenty of extras: auto form fillers, tabbed browsing, desktop search, spyware blockers, translators, spell checkers, and so on. I’m not going to describe all these tools, as they aren’t directly related to SEO, but they’re definitely useful.

You really don’t need all of them, but hey, here they are if you really want to experiment. You can find these toolbars here:

toolbar.google.com

toolbar.yahoo.com

toolbar.msn.com

toolbar.ask.com

Geek or no geek

Many readers of this blog’s first edition are business people who don’t plan to do the search-engine work themselves (or, in some cases, realize that it’s a lot of work and need to find someone with more time or technical skills to do the work).

However, having read the blog,and are in a better position to find and direct someone. As one reader-cum-client, told me, “There’s a lot of snake oil in this business,” so his reading helped him understand the basics and ask the right questions of search-engine optimization firms.they understand far more about search engines.Unfortunately, all these toolbars require Microsoft Windows and Internet Explorer; that’s most of you but, I realize, not all. You can see these toolbars, along with the Alexa toolbar, in Figure 1-4. Don’t worry, you don’t have to have all this clutter on your screen all the time. Right-click a blank space on any toolbar, and you can add and remove toolbars temporarily; simply open a toolbar when you need it. I leave the Google toolbar on all the time, and open the others now and then.One thing I do find frustrating about these systems is the pop-up blockers. Yes, they can be helpful, but often they block pop-up windows that I want to see; if you find that you click a link and it doesn’t open, try Ctrl+clicking (which may temporarily disable the pop-up blocker), or disable the blocker on the toolbar.I refer to the Google toolbar here and there throughout this blog because it provides you with the following useful features:A way to search Google without going to www.google.com firstA quick view of the Google PageRank, an important metric that I explain in blog 14

A quick way to see if a Web page is already indexed by Google A quick way to see some of the pages linking to a Web pageThe toolbar has a number of other useful features, but the preceding features are the most useful for the purposes of this blog. Turn on the Info button after installing the toolbar:

1. Click the Options button.

2. Click the More tab in the Toolbar Options dialog box. 3. Enable the Page Info checkbox and click OK.


Alexa toolbar

Alexa is a company owned by Amazon.com, and a partner with Google and Microsoft. It’s been around a long time, and millions

of people around the world use it. Every time someone uses the toolbar to visit a Web site, the toolbar sends the URL to

Alexa, allowing the system to create an enormous database of site visits. The toolbar can provide traffic information to you;

you can quickly see how popular a site is and even view a detailed traffic analysis, such as an estimate of the percentage of

Internet users who visit the site each month.

Work with the Alexa toolbar for a while, and you’ll quickly get a feel for site popularity. A site ranks 453? That’s pretty

good. 1,987,123? That’s a sign that hardly anyone visits the site. In addition, it provides a quick way to find infor- mation

about who owns the site on which the current page sits, and how many pages link to the current page. You can find the Alexa

toolbar, shown in Figure 1-4, at download.alexa.com .

I’ve been criticized for recommending the Alexa toolbar in the first edition of this blog: Many people claim it is spyware.

Some anti-spyware programs search for the toolbar and flag it as spyware, though others don’t. As I men- tioned, the toolbar

sends the URLs you’re visiting. However, Alexa states on the site (and I believe them) that “The Alexa Toolbar contains no

advertising and does not profile or target you.” I know for sure that it doesn’t display ads. Alexa doesn’t steal your

usernames and passwords, as is occasionally claimed. Alexa does gather information about where you visit, but it doesn’t know

who you are, so does it matter? Decide for yourself.

Your One-Hour Search-Engine- Friendly Web Site Makeover

In This blog

Finding your site in the search engines Choosing keywords

Examining your pages for problems

Getting search engines to read and index your pages

few small changes can make a big difference in your site’s position in the search engines. So rather than force you to read

this entire blog before you can get anything done, this blog helps you identify problems with your site and, with a little

luck, shows you how to make a significant dif- ference through quick fixes.

It’s possible that you may not make significant progress in a single hour, as the blog title promises. You may identify

serious problems with your site that can’t be fixed quickly. Sorry, that’s life! The purpose of this blog is to help you

identify a few obvious problems and, perhaps, make some quick fixes with the goal of really getting something done.

Is Your Site Indexed?

It’s important to find out if your site is actually in a search engine or direc- tory. Your site doesn’t come up when someone

searches at Google for rodent racing? Can’t find it in the Yahoo! Directory? Have you ever thought that per- haps it simply

isn’t there? In the next several sections, I explain how to find out if your site is indexed in a few different systems.

Some of the systems into which you want to place your Web site aren’t household names. If I mention a search system that you

don’t recognize, page back to blog 1 to find out more about it.

Investigating Search Engines and Directories

The term search engine has become the predominant term for search system or search site, but before reading any further, you need to understand the dif- ferent types of search, um, thingies, you’re going to run across. Basically, you need to know about four thingies.

Search indexes or search engines

Search indexes or engines are the predominant type of search tools you’ll run across. Originally, the term search engine referred to some kind of search index, a huge database containing information from individual Web sites.

Large search-index companies own thousands of computers that use soft- ware known as spiders or robots (or just plain bots ) to grab Web pages and read the information stored in them. These systems don’t always grab all the information on each page or all the pages in a Web site, but they grab a signif- icant amount of information and use complex algorithms — calculations based on complicated formulae — to index that information. Google, shown in Figure 1-1, is the world’s most popular search engine, closely followed by Yahoo! and MSN.

Index envy

Late in 2005, Yahoo! ( www.yahoo.com ) claimed that its index contained information about almost 20 billion pages, along with almost 2 billion images and 50 million audio and video pages. Google ( www.google.com ) used to

actually state on its home page how many pages it indexed — they reached 15 billion or so at one point — but decided not to play the “mine is bigger than yours” game with Yahoo!
 
Search directories

A directory is a categorized collection of information about Web sites. Rather than containing information from Web pages, it contains information about Web sites.

The most significant search directories are owned by Yahoo! ( dir.yahoo. com ) and the Open Directory Project ( www.dmoz.org ). (You can see an example of Open Directory Project information, displayed in Google — dir.google.com — in Figure 1-2.) Directory companies don’t use spiders or bots to download and index pages on the Web sites in the directory; rather, for each Web site, the directory contains information, such as a title and description, submitted by the site owner. The two most important directo- ries, Yahoo! and Open Directory, have staff members who examine all the sites in the directory to make sure they’re placed into the correct categories and meet certain quality criteria. Smaller directories often accept sites based on the owners’ submission, with little verification.

Here’s how to see the difference between Yahoo!’s search results and the Yahoo! directory:

1. Go to www.yahoo.com .

2. Type a word into the Search box. 3. Click the Search button.

The list of Web sites that appears is called the Yahoo! Search results, which are currently provided by Google.

4. Notice the Directory tab at the top of the page.

You see a line that says something like Category: Footwear Retailers. You also see the line underneath some of the search results.

5. Click either the tab or link.

You end up in the Yahoo! directory. (You can go directly to the directory by using dir.yahoo.com .)

Non-spidered indexes

I wasn’t sure what to call these things, so I made up a name: non-spidered indexes. A number of small indexes, less important than major indexes such as Google, don’t use spiders to examine the full contents of each page in the index. Rather, the index contains background information about each page, such as titles, descriptions, and keywords. In some cases, this information comes from the meta tags pulled off the pages in the index. In other cases, the person who enters the site into the index provides this information.

Pay-per-click systems

Some systems provide pay-per-click listings. Advertisers place small ads into the systems, and when users perform their searches, the results contain some of these sponsored listings, typically above and to the right of the free listings.


Keeping the terms straight

Here are a few additional terms that you will see scattered throughout the blog:

Search site: This Web site lets you search through some kind of index or directory of Web sites, or perhaps both an index and directory. (In some cases, search sites known as meta indexes allow you to search through multiple indices.) Google.com, AOL.com, and EarthLink.com are all search sites. DogPile.com and Mamma.com are meta-index search sites. Search system: This organization possesses a combination of software, hardware, and people that indexes or categorizes Web sites — they build the index or directory you search through at a search site. The distinction is important, because a search site may not actually own a search index or directory. For instance, Google is a search system — it displays results from the index that it creates for itself — but AOL.com and EarthLink.com aren’t. In fact, if you search at AOL.com or EarthLink. com and search, you actually get Google search results.

Google and the Open Directory Project provide search results to hun- dreds of search sites.

Search term: This is the word, or words, that someone types into a search engine when looking for information.

Search results: Results are the information returned to you (the results of your search term) when you go to a search site and search for some- thing. As just explained, in many cases the search results you see don’t come from the search site you’re using, but from some other search system.

Natural search results: A Web page can appear on a search-results page two ways: The search engine may place it on the page because the site owner paid to be there (pay-per-click ads), or it may pull the page out of its index because it thinks the page matches the search term well. These free placements are often known as natural search results ; you’ll also hear the term organic and sometimes even algorithmic .

Search engine optimization (SEO) : Search engine optimization (also known as SEO ) refers to “optimizing” Web sites and Web pages to rank well in the search engines . . . the subject of this blog, of course.

Why bother with search engines?

Why bother using search engines for your marketing? Because search engines represent the single most important source of new Web site visitors.

You may have heard that most Web site visits begin at a search engine. Well, this isn’t true. It was true several years ago, and many people continue to use these outdated statistics because they sound good — “80 percent of all Web site visitors reach the site through a search engine,” for instance. However, in 2003, that claim was finally put to rest. The number of search-originated site visits dropped below the 50-percent mark. Most Web site visitors reach their destinations by either typing a URL — a Web address — into their browsers and going there directly or by clicking a link on another site that takes them there. Most visitors don’t reach their destinations by starting at the search engines.

However, search engines are still extremely important for a number of reasons:

The proportion of visits originating at search engines is significant. Not so long ago, one survey put the number at almost 50 percent. Sure, it’s not 80 percent, but it’s still a lot of traffic.

According to a report by eMarketer published early in 2005, 21 percent of American Internet users use a search engine four or more times each day; PEW Internet estimated that 38 million Americans use search engines every day.

A study by iCrossing in the summer of 2005 found that 40 percent of people do online research prior to purchasing products.

Of the visits that don’t originate at a search engine, a large proportion are revisits — people who know exactly where they want to go. This isn’t new business; it’s repeat business. Most new visits come through the search engines — that is, search engines are the single most impor- tant source of new visitors to Web sites.

Some studies indicate that a large number of buyers begin at the search engines. That is, of all the people who go online planning to buy some- thing or looking for product information, perhaps over 67 percent use a search engine, according to a study in 2005 by iCrossing.

The search engines represent a cheap way to reach people. In general, you get more bang for your buck going after free search-engine traffic than almost any other form of advertising or marketing.

Where Do People Search?

You can search for Web sites at many places. Literally thousands of sites, in fact, provide the ability to search the Web. (What you may not realize, how- ever, is that many sites search only a small subset of the World Wide Web.)

However, most searches are carried out at a small number of search sites. How do the world’s most popular search sites rank? That depends on how you measure popularity:

Percentage of site visitors (audience reach) Total number of visitors

Total number of searches carried out at a site

Total number of hours visitors spend searching at the site

Each measurement provides a slightly different ranking, though all provide a similar picture, with the same sites generally appearing on the list, though some in slightly different positions.


The following list runs down the world’s most popular search sites, based on one month of searches during 2005 — 4.5 billion searches — according to a Nielsen/NetRatings study. These statistics are for U.S. Internet users:

Google.com Yahoo.com
 46.2% 22.5%

MSN.com AOL.com My Way
 12.6%

5.4% 2.2%

Ask (AskJeeves) Netscape.com
 1.6% 1.6%

iWon
 0.9%

Earthlink DogPile Others
 0.8% 0.9% 5.3%

Remember, this is a list of search sites, not search systems. In some cases, the sites have own their own systems. Google provides its own search results, but AOL doesn’t. (AOL gets its results from Google.)

The fact that some sites get results from other search systems means two things.

The numbers in the preceding list are somewhat misleading. They sug- gest that Google has around 46.2 percent of all searches. But Google also feeds AOL its results — add AOL’s searches to Google’s, and you’ve got 51.6 percent of all searches. In addition, Google feeds Netscape (another 1.6 percent according to NetRatings) and EarthLink (0.8 percent ). And DogPile is a meta search engine: Search at DogPile, and you see results from Google, Yahoo!, MSN, and Ask.

You can ignore some of these systems. At present, for example, and for the foreseeable future, you don’t need to worry about AOL.com. Even though it’s one of the world’s top search sites, you can forget about it. Sure, keep it in the back of your mind, but as long as you remember that Google feeds AOL, you need to worry about Google only.

Now reexamine the preceding list of the world’s most important search sites and see what you remove so you can get closer to a list of sites you care about.


The Top Search Sites:

Search Site

Google.com

Yahoo.com

MSN.com

AOL.com

MyWay.com

Ask.com (also known as AskJeeves.com)

Netscape.com

iWon.com

EarthLink.com

DogPile.com


To summarize, five important systems are left:

Google Yahoo MSN Ask

Open Directory Project

That’s not so bad, is it? You’ve just gone from thousands of sites down to five. Note, by the way, that the top three positions may shift around a little. Google has already lost a large proportion of its share (when I wrote the first edition of this book Google had around three quarters of the market . . . now it’s probably a little over one half), and a big battle’s brewing between the top three; in fact 2006 is turning out to be the year of bribing people to bring them to search. Take a look at Google partner Blingo ( www.blingo.com ) and at MSN Search and Win ( www.MSNSearchAndWin.com ).

Now, some of you may be thinking, “Aren’t you missing some sites? What hap- pened to HotBot, Mamma.com, WebCrawler, Lycos, and all the other systems that were so well known a few years ago?” A lot of them have disappeared or have turned over a new leaf and are pursuing other opportunities.

For example, Northern Light, a system well known in the late 1990s, now sells search software. And in the cases in which the search sites are still running, they’re generally fed by other search systems. Mamma.com, DogPile, and MetaCrawler get search results from the top four systems, for instance, and HotBot gets results from Ask. Altavista and AllTheWeb get their data from Yahoo! If the search site you remember isn’t mentioned here, it’s either out of business, being fed by someone else, or simply not important in the big scheme of things.