Monday, 12 December 2011

Listing Dynamic Web Pages in Search Engines


Problems With Dynamic Languages

Most search engines do not like to follow java script or spider deeply into dynamic driven websites. Since scripts can create millions and millions of pages on the fly search engines need to ensure some level of quality before they will want to list all the pages.

Session ID's & Cookies

Search engines do not like reading session IDs or cookies. When a search engine is given a session ID they will usually leave the site. If spiders indexed websites that offered the spider a session ID it would frequently cause the spiders to overload the server. Session ID's would also make the site seem many times larger than it is.

General Dynamic Page Listing Tips

Search engines are getting better at listing dynamic web pages. There are a few basic thumb rules to help search engines index your dynamic website.
  • Build a linking campaign. As you get more inbound links search engine spiders will have more reason to trust the quality of your content and they will spider deeper through your site.
  • Use short query strings. Most search engines will do well to list dynamic pages if each query string is kept less than 10 digits.
  • Minimize the number of variables. When possible you want to use three or less different parameters, the fewer the better. If you use long parameter strings and a half dozen parameters it is a fair bet that the site will not get indexed.
  • If you still have problems you can use a CGI script to take the query string out of your dynamic content or try one of the other dynamic page workarounds listed below.
  • Other dynamic workarounds: There are multiple ways to list dynamic web pages in search engines. Common techniques are:
    • Hard coded text links
    • Bare bone pages
    • Coldfusion site maps
    • Apache Mod Rewrite
    • IIS Rewrite Software
    • Paid Inclusion

Hard Coded Text Link

To list dynamic web pages in search engines you can capture the entire url in a link like:
<a href="http://www.search-marketing.info/catalog.html?item=widget&color=green&model=6">Green widgets style 6</a>
If you use static links like listed above to reference dynamic pages, search engines will usually be able to index them. Many sites use a site map which captures the most important interior pages.
If you have enough link popularity and link into a couple of your main category pages using static links then search engines will usually be able to index the rest of the site.

Bare Bones Pages

You also can make a bare bones static page for each one you want listed in search engines.
<html>
<head>
<title>Green Widgets style 6</title>
</head>
<body>
<!--#exec cgi="myscript.pl?greenwidget-6"-->
</body>
</html>

Coldfusion Site Map

For ColdFusion you can code a site map page using a code similar to the following.
<cfquery name="getPages" datasource="myDatasource">
SELECT *
FROM pages
WHERE status = 1
ORDER BY sortorder
</cfquery>
<cfoutput query="getPages">
<a href="index.cfm?pageID=#getPages.pageID#">#getPages.pageName#</a>
</cfoutput>

Apache Mod Rewrite

For Apache servers there is a way to make dynamic pages seem like static pages called Mod Rewrite. The documentation on MOD Rewrite is located on the Apache website. Apache MOD Rewrite.

IIS Rewrite Software

IIS servers do not have the built in rewrite features like Apache Mod Rewrite. You still can rewrite your URL's on IIS servers using custom built software programs.

Trusted Feed Paid Inclusion

You can pay to use trusted feeds to upload dynamic content to Yahoo! to acquire traffic at a cost per click bases through the Overture Site Match program. I would consider this a last option for many sites since many business models can not support the incremental cost per click charges.

1 comment:

  1. This comment has been removed by a blog administrator.

    ReplyDelete