SEO Guidelines for Developers
This is the developer portion of my SEO tutorial. If you have not read the introduction then please read that first.
Table of Contents
As a developer it is your job to understand the technical side of the web. You must take into account many different factors that will affect the way visitors use your software. With the vast number of things you must worry about just to get your project working and out the door, SEO can often be a thorn in your side. Some techniques that make a site more functional and easy to use for a person will make it impossible for a search engine to use. I won't go into too much detail but AJAXing your websites is just one example of a technology that is amazingly slick for the end user but completely breaks down when a search engine tries to crawl your page unless you've taken that into consideration and worked around it. The point is that you must put thought into SEO from the very beginning.
You do have to think a little about the technical details of how a search engine will index your content, but ultimately as long as your are building an application that is capable of allowing your client(s) to provide up-to-date and relevant content then you are on the right track. For a developer truly organic website exposure simply means that the client has total control over the site's content and accessibility. The nature of the content will depend on what your client wants to achieve. You must work very closely with your client to obtain a clear picture of his/her needs and structure your software to facilitate those needs. If your client wants to blog about his/her industry, provide an easy to use blog administration interface that is intuitive and immediately visible to them.
An example of an intuitive interface, in my opinion, would be an interface that seemlessly integrates with the visitor version of the site. In other words, while your client is viewing content that their visitors would normally see, all he/she should have to do is login with their username and password and suddenly the content transforms to expose some new functions only available to the logged in user. Most websites do this these days. Facebook is a prime example. You can view someone else's profile page and see the content that user has provided; if the owner of the profile is logged in and views their own profile page, they will have access to several "edit" links that you would not have seen. If your automatic response is "duh!" in regards to building such an intuitive interface then that is great! However, if your first instinct is to build a "control panel" from which the client can "administer" the site, then you may want to rethink your design a little bit.
Occasionally you have so many options and functionality that you must provide an administrator area. That is fine, but it is still a good practice to provide a slimmed down version of those tools (or shortcuts to those tools) in a more intuitive way. When the client goes to his blog and then decides to create a new post, don't make them fumble around through an administrator control panel to find the blog manager. Put a "new post" link on the blog index page that only shows up when your client is logged in. Simple niceties like this make all the difference.
This discussion might not seem like it is related to SEO, but it most definitely is. It's not your job as the developer to deliver good content. It is your job to deliver a product capable of delivering good content. The tools that you build for your client are the foundation for good SEO. Making them easy to use will make your client's life easier by not forcing them to think too much about how to even use their new web application. When you first go to Facebook it takes a little getting used to, but pretty much everything is easy to find and right where you would expect it to be.
Often-times when I take a step back I realize that I have no need for a "management" page. Take this blog for instance; when I first built CodeTunnel I had every intention of building a blog post management page. It would have been an admin-only page with a list of blog titles that I could click on. From there I would have been able to add, edit, and delete blog posts. When I was done "administering" my blog I would then go to the regular content pages and view my changes the way readers would see them. After taking a step back I realized that all that infrastructure was pointless. I would be duplicating a list of blog posts when the blog index that readers see when they first come to the site was already there! I was also going to build a search function on this management page until I realized that too was pointless since I wanted to build a search tool for visitors as well.
In the end I put a "New Post" button at the top of the blog index, "edit" and "delete" links on every blog post summary on the blog index and on each individual blog page. It was far less work, and way more intuitive. Now if I'm reading a post and decide I need to change something I just click "edit", whereas I would have had to find my administrator control panel, find my blog post manager, find my blog post all over again, then edit it. That would not have been very intuitive at all.
White Hat vs Black Hat
The entire white hat vs black hat discussion can be boiled down to good vs evil. White hat techniques in SEO are like tax deductions while black hat techniques are like tax evasion. White hat techniques are anything that you can do to genuinely and honestly increase exposure to your site. Years ago this was an important discussion because search engines were relatively archaic compared to today's search engines. There were specific things you could do to "trick" the search engines into up-ranking your site. There was a dividing line between which techniques were considered to be well-intended and which techniques were considered cheating. Today, pulling off a black hat technique would require just as much or more work than just building traffic the honest way. Search engines have gotten far too smart for you to take advantage of them with black hat.
Black hat developers used to build sites called link farms. These sites used to link to target sites because search engines would measure back-links. Back-links are links from other sources on the web to your content. These things are still taken into account today but the process that occurs when ranking sites is so complex that it's very hard to game. Link farms get shut down faster than they go up. There really is no such thing as a successful black hat technique anymore. Stuffing your site with keywords will not help you either. Black hat developers would fill a page with as many keywords as they could. They would sometimes make the entire background of a site contain giant walls of text that were the same color as the background and would just be huge lists of keywords. In this way they were able to trick search engines into thinking some content was relevant based on these keywords that only the search engine would see.
Even if you are successful in gaming the system, it won't last long. Once you are discovered you will be shut down and your site will be blacklisted. Once you are on Google's blacklist, it's game over for organic traffic. You cannot fight the system because you will not win. Do not even consider attempting to game the system unless you want to get nowhere fast. Working and flowing with the system is the only way to gain any benefit from it.
Other Considerations
Providing genuine and relevant content to visitors is the number one goal. Always stop and take a step back to consider the intuitiveness of the application you are developing. While an intuitive design is important in order to give your client the right tools, there are a couple technical obstacles that do make a difference.
Readable URLs
I am an avid consumer of Microsoft development products. I design applications on the .NET platform every day. Not too long ago there was only one technology for web development in .NET and that was ASP.NET Web Forms. If you have ever developed a web forms application then you already know what a pain it is to design clean URLs in a web forms application. The introduction of .NET 4 and ASP.NET MVC have made URL routing and design a breeze, but before that it was almost impossible. There are still many web technologies that have ugly URLs with page extensions like .aspx, .html, .php, etc. Have a look at this URL:
While I'm sure "/dp/1430225491/ref=sr_1_1?ie=UTF8&qid=1311693861&sr=8-1" means a lot to you as the developer, it doesn't mean squat to your client. Likewise, it doesn't mean squat to search engines either. A search engine will index this URL just fine, but the URL does almost nothing to help out the search engine. It does not describe the content at all after about half way through. I know that in development you must have access to certain information such as unique ID numbers and the like, but there are better ways to get what you want and give the search engine what it wants.
Take a look at the URL for this very blog post. Pretty clean if I do say so myself.
http://www.CodeTunnel.com/blog/post/86/seo-guidelines-for-developers
Obviously a URL can't read like plain English, but you can come close. You are on my blog, reading post number 86, with the title "SEO Guidelines for Developers". It makes sense to readers and it does a good job of describing the content so that search engines can use it when evaluating searches by users. Obviously I have a need to know exactly which blog post to load when the reader clicks through. The 86 in the above URL is a unique ID number for this blog post and it has to be there. If it wasn't there then I would have to ensure that every title of every blog post was absolutely unique and that would be a pain as I write more and more.
Technically the title portion at the end is unnecessary. If you removed it you would still pull up the right blog post because all I really need is the ID number. If you navigate to http://www.CodeTunnel.com/blog/post/86 you will actually be redirected to the full URL with the title included. The title is so useful to readers and search engines in a URL that I ensure it is there even though I as a developer could care less if the title is in the URL. Pasting http://www.codetunnel.com/blog/post/86/seo-guidelines-for-developers into an instant message or email means so much more to the recipient than http://CodeTunnel.com/blog/post/86 despite the fact that the URL is shorter.
Take care in designing your URLs if your development platform allows it. If you are unable to affect the craziness of your URLs then don't worry too much as it is only a small portion of the overall application. URLs are important, but they are less important than relevant content. If you can't modify your URL structure easily then focus your efforts on making sure the content they serve up is worthwhile.
Utilize Progressive Enhancement
Javascript is an amazing tool for a developer. With javascript one can make web applications do some pretty amazing things. Being able to load content via AJAX is amazingly useful. If you have javascript enabled and you click through the pages in the blog index for Code Tunnel you will notice a spiffy animation as the site loads the pages you request. This is pretty neat for the end user but not so neat for a search engine. When Google crawls my blog it does not execute any javascript. It simply looks for links that it can find and then moves onto the pages that are linked. If the only way you could get between pages was via that javascript animation then Google would never be able to index any content after the first page.
Progressive Enhancement is the practice of designing a site to work without any javascript enhancements and then enhancing the user experience when they do have javascript enabled. It is a little more work than just throwing out some javascript but it is very important if you want a search engine to see content. If you turn off javascript, reload this blog and navigate through the pages you will notice that the pages still load fine. You get a full page refresh instead of a fancy animation but that is hardly an issue as long as it works. There are some sites that work entirely using javascript and then they wonder why none of their content is being indexed by search engines.
Conclusion
Aside from a couple specific technical blunders to avoid, it really is all about an intuitive user interface that allows the client to maintain and update his application. The ability to quickly and easily feed in new content will be your client's greatest asset in his SEO efforts. Be a white hat developer, iron out technical hurdles, and design a clean intuitive user interface.