HTML for Absolute Beginners, by Jon Storm.
Go to : Next (And finally...); Previous (Uploading it to the internet); Beginner's HTML Index; Jargon dictionary; Ultraspace; Jon Storm.

Getting noticed : search engines and META tags

When we looked at the basic structure of a webpage, we skipped right past the area between the <HEAD> and </HEAD> tags. You don't have to put anything in this area, but there are several things that you really don't want to miss out. Why? In a word (oh alright, two words) : search engines.

If you want people to admire your site, they have to be able to find it, and that's what search engines are for. Say your particular passion is, oh, ferret racing, and that's what your site is about. Your target audience, naturally enough, is other ferret racing enthusiasts. So if someone goes to a search engine and does a search for "ferret racing", it would be nice if the search brought up your site, preferably on the first page of results and with an appealing description of its merits. The key is getting properly indexed, and there are several tags you can put into the HEAD area to help search engines index and list your site better.

Most search engines use programs called "spiders" or "bots" to index websites, with little or no human input. They have to do this to have any chance of being at all comprehensive, because there are literally millions of sites out there, and more being created all the time. But spiders aren't very bright, and unless you help them they are liable to list your site under its first few lines of text, regardless of what they are. Or even just as "untitled", which isn't likely to bring many visitors in.

The first thing you need to add, and the most important, is a title. The HTML tag for the title of this page is :

<TITLE>Beginner's HTML - Search engines & meta tags</TITLE>

Like all the tags we are discussing in this section, it goes between the <HEAD> and </HEAD> tags, usually as the first item. If you include a title, almost all search engines will use that as the main caption for your site in their listing, so go for something descriptive and not too long. (And not "my first webpage". Please.) Some search engines use the text of your title as a major indexing key, too, so don't put exactly the same title on every page. The title of the page currently being displayed also appears in the title bar (usually blue) at the top of most browsers.

The other tags you can use to help get indexed properly are called META tags. There are a whole group of these, most of which don't do anything very important, but two of them are very useful. The first lets you provide a brief description of what your site is about, and looks like this :

<META NAME="Description" CONTENT="A set of HTML tutorials for absolute beginners. This page is about getting indexed by search engines">

Note that the whole content goes inside the tag, and there is no "off" tag. As with other tags, it doesn't matter whether you put the tag (or its attributes) in upper case, lower case or a mixture of both.

When someone does a search, a search engine generally finds hundreds of sites, and lists the first dozen or so that it thinks match the search criteria best, with a caption and a brief description. Your title provides the caption, and your meta description tag provides the description. If you don't have a title search engines usually just use your URL or "untitled" for the caption, and if there is no meta description tag they tend to use the first twenty to twenty-five words of text on your site, regardless of what they are. Looks awful. So make sure you always put in a meta description tag. You should keep it under twenty-five words, or they are liable to cut it off in mid-sentence.

Another useful tag is meta keywords. This gives a list of keywords you would like your page to be indexed under, and looks something like this :

<META NAME="KeyWords" CONTENT="html, webpage, website, tutorial, instructions, beginners, dummies, simple, meta, search engine, search engines">

(No prizes for guessing that that is the meta keywords tag for this page). Unfortunately this tag has been heavily abused, and so some search engines now ignore it, but it is still worth including for those that do honour it. Try to think of words that people who might enjoy your site might search for, and list them as I have here, most important first. If there is an important word that describes your site with more than one spelling, list both.

Don't include words that have nothing to do with your site, in hope of getting found by more searches. If you do, you may find yourself getting put to the bottom of search engine rankings rather than the top, or even excluded altogether. Search engines are wise to this trick and it really annoys them. And anyway, think about it. Suppose, knowing that lots of people hunt for smut, you put a few words like "sex" "nude" and so forth in the meta keywords tag of your ferret-racing site. Do you really think people hunting for smut are going to be happy to arrive at a ferret-racing site instead? Doesn't seem likely, does it? They will just curse you (and the search engine) for misleading them, and go somewhere else.

Unfortunately it's hard to advise you exactly what works well in a meta keywords tag, because it depends on the individual search engine, and they don't like to go into too much detail about their indexing methods because of abuse by spammers.

Most other meta tags don't actually do anything very useful. If you use an editor to create your HTML, chances are it will add a couple of meta tags giving itself credit for it. Nothing bad will happen if you delete these tags, but if you like your HTML editor you can leave them in there - they make no difference either way. You also see meta tags listing the author of a document, and a whole host of other things. You can safely ignore the lot, apart from the Description and Keywords tags.

However, there are a couple of other meta tags you may occasionally find useful. One rather impressive one can automatically redirect visitors to another page - very handy if you are moving to a different webhost. You can add it to pages at the old address, so that visitors are automatically sent to the new site. A tag to redirect a visitor to the Jon Storm index page would look like this :


The 5 in this example means that there will be a five second delay before the visitor is redirected (change the number for a longer or shorter pause), enough for them to read a short message, such as "This site has moved. You will be taken there automatically in a few seconds - please update your bookmarks". The five seconds starts once the page has completely finished loading, so if it's the original page with the tag added, your visitor may be twiddling their thumbs for a while before they get redirected, especially if it has lots of graphics on it. So don't use the original page, do a new one with just the "you are being redirected" message. It's a good idea to include a link to the new site as well, both in case the redirect fails for some reason, and also to help search engines find the new site - they don't usually follow automatic redirects.

Another meta tag that is sometimes useful allows you to tell a search engine not to index your site - maybe you are testing it and it's not ready for the world yet, or maybe you are moving and don't want it indexed under the old address. This is the meta robots tag, which goes like this :


As well as NOINDEX, which is pretty self-explanatory, you can put NOFOLLOW, which tells a search engine bot or spider not to follow links on the page - if you want to put both, separate them with a comma. Alternatively you can put ALL, meaning that you do want the page indexed, but it's a bit pointless as the spider will do it anyway.

Getting listed

If you do nothing at all, your site will probably get indexed by some of the search engines - eventually. Some of them routinely spider all the homepages hosted by the major ISPs once in a while, so they may find you that way. But if you want to get listed before the end of time, you need to tell them you are there.

There are literally thousands of search engines, but fortunately you don't need to visit them all. The vast majority of searches are done on only a few of them, so you can concentrate your efforts on those. Which ones are most important is largely a matter of opinion, but most people's lists would include, in no particular order, Google, Alta Vista, Excite, Yahoo, and Lycos. All you have to do is visit each site in turn, look for a link that says "submit your site" or "Add URL", and follow the instructions. Do take the time to read the help provided on each site, because it can often help you to get a better listing.

Bear in mind that the top search engines have a worldwide audience. If your site is tied to a particular geographical area (say it is promoting a restaurant in London, for example), visitors to your site from the USA or India aren't likely to drum up much extra business. Several of the majors operate regional subsidiaries : in the UK, for example, both Excite and Yahoo have UK-specific sites, at Excite UK and Yahoo UK & Ireland respectively. You might do better to register with those, instead of the global parent.

It's also a good idea to register with a few sites that specialise in your country or region, even if your site isn't particularly region-specific : UKPlus in the UK for example. And with any sites you run across that specialise in your particular area of expertise.

Search engines often take many weeks to process your submission (particularly ones that use human indexers, such as Yahoo), so you do have to be patient. Don't hassle them, or bombard them with multiple submissions : you are more likely to get yourself blacklisted than listed if you annoy them. If your listing still hasn't appeared after a couple of months, or the period specified on their submission page, then resubmit it, perhaps rephrased. But once your listings do start coming through, after a while you may well find that you are also appearing in engines you didn't submit to. Most search engines feed off each other's listings to some extent, so you can sometimes get a sort of chain reaction.

While you are doing the rounds of the search engines, you will run into a number of submission services. Typically they will offer to submit your site to hundreds of search engines automatically, usually for a fee. But I don't think it's worth it. You are much more likely to get a good listing if you personally tailor your submission for the engine you are submitting it to. And some search engines don't accept automatic submissions, or give them very low priority.

Most searches turn up dozens if not hundreds of sites, and you are much more likely to get visitors if you can get a high ranking, so you appear near the top of the list. Search engines tend to be very cagey about exactly how their ranking systems work, but they often take into account how many links there are to your site from other sites, so it's a good idea to swap links with people running similar sites to yours.

Getting noticed

Of course, search engines aren't the only way to attract visitors. In many ways, the best way to publicise a website is outside cyberspace, in the boring old real world (remember reality? You must have been there once). Printing your URL (website address) on things and sending or giving them to people is one good way of getting visitors. Another is getting your site reviewed in a magazine or newspaper, or if you are really lucky, on TV. Make sure that your site works properly throughout first though, with no dead links, typing errors, or graphics that don't display, or you may get a review that says your site is rubbish, which isn't likely to help. Get a few friends to check it out and criticise it thoroughly before you submit it for review - they are bound to notice things you haven't. Fixing the bugs after the reviewer has left is too late - they aren't likely to come back.

Back in cyberspace, how about including your URL in your sig (email/news signature) whenever you send an email, or post to a newsgroup? Don't post adverts for your site in newsgroups where it isn't relevant, though, or send bulk emails promoting your site to people you don't know. Seriously. This is called spamming, and internet users absolutely hate it. They will report you to your ISP, who will very likely terminate your account on the spot. If you don't believe me, go to your ISP's website and read their AUP (Acceptable Use Policy) document.

A website is a great publicity tool, but like most publicity tools it works better if it's part of the publicity campaign, rather than all of it. So don't expect the search engines to make you famous on their own, but they can certainly help.

Go to : Next (And finally...); Previous (Upload); Beginner's HTML Index; Jargon dictionary; Ultraspace; Jon Storm.