DNS and SEO

tags: #tech, #hosting, #seo, #blog, #webdev, #cloudflare

You may have noticed I decomissioned my old Hugo site and switched this new blog over to my custom domain (adavidwilson.com). Doing so reminded me of something I looked into awhile back regarding hosting on a domain’s apex/root or on a subdomain like www. These days, it’s hard to realize when sites are still hosted using the traditional subdomain approach if they’re using www, since browsers tend to hide that now unless you explicitly check it in the address bar, but there are some reasons to choose the longer URL over using the root. From my basic research, one thing is certain: don’t configure both independently if they’re serving the same content!

DNS Configuration and HTTP Redirects

Before continuing, I should probably lay out a few of the basic concepts:

  • Domains are governed by authorities that constitute the hierarchy of providers for the Domain Name System (DNS).
  • Domains are essentially tables in databases of records that define how requests to the domain should be resolved by clients.
  • Domain records have types that establish certain properties. For a simple static blog, these records are most important:
    • A: The “address” record. It maps hostnames to IPv4 addresses.
    • AAAA: Same as A records but for IPv6.
    • CNAME: A “canonical name” record. It maps hostnames to other hostnames.
  • The “apex” or “root” of a domain is symbolized with @. Most providers will allow you to leave the name blank for this.

Besides these DNS concepts, there are other things in HTTP called redirects that help. For our purposes, we’ll only care about 301 Moved Permanently.

So, there are a few ways to configure your site. Assume your site (mysite.com) is hosted from a single IPv4 address for simplicity. If you have IPv6, just duplicate the address records using AAAA. Say that IP address is 76.216.17.2. Here are some (nowhere near exhaustive) configurations:

  1. A simple A record pointing @ to 76.216.17.2.
    • This will serve your site from mysite.com, but requests to www.mysite.com will fail.
  2. The same A record as in 1 plus a CNAME pointing www to @.
    • This will serve your site both from mysite.com and www.mysite.com assuming your server at 76.216.17.2 is configured to respond to requests from both and your certificates are configured for both hostnames.
  3. A simple A record pointing www to 76.216.17.2.
    • This will serve your site from www.mysite.com, but requests to mysite.com will fail.
  4. The same A record as in 3 plus a CNAME pointing @ to www.
    • This will serve your site from www.mysite.com and mysite.com assuming your server is configured like in 2.
  5. Two A records:
    • One pointing @ to either 76.216.17.2 or some other server’s IP;
    • Another pointing www to either 76.216.17.2 or some other server’s IP.

Now, note in all of these cases, DNS just tells a client what IP to hit if they use a particular domain name. It’s up to the server at the IP the name resolves to on how it handles each of the different hostnames. You could have the same server return different web pages for different hostnames, or redirect names to others. That’s where the 301 Moved Permanently HTTP response comes in. Let’s say you go with either option 2 or 4 above. Now, both mysite.com and www.mysite.com will resolve to your server living at IP address 76.216.17.2. As mentioned, you could have your server return the same or different content for those two hostnames as a standard 200 OK. The problems or at least the considerations you need to be aware of with this approach are detailed below.

Alternatively, you can have your server redirect one of them to the other using a 301 or other redirection responses (302, 307, 308). I have my site configured to terminate in Cloudflare with an A record and redirect to www using a rule that corresponds to what you’d ordinarily configure your HTTP server to do with a 301. My record for www is a special “worker” type unique to Cloudflare, but I suspect it’s either a CNAME or A under the hood. This is roughly equivalent to either option 2 or 5 depending on how Cloudflare resolves the worker record. Either way, if someone hits @, Cloudflare redirects them to www with a 301, and www works as a direct path to the site.

SEO Optimization Considerations and Other Trade-Offs

So, why would you want a setup like this? If your server responds to both @ and www serving the same content, search engines will not care and treat each as distinct sites. So any metrics you get will be divided between the two hostnames as though they were two separate sites. In addition, choosing to use @ instead of www to serve your site has some restrictions on what you can do with CDNs and caching. In brief, using www especially with a CNAME record gives you the most flexibility while retaining SEO optimization as long as you avoid pretending both hostnames are the same site by serving the same content on both. If both hostnames really are distinct sites, then set it up however you want: two different server IPs, or two sites served by the same server. At the same time, if convenience is all you’re after and you don’t need the flexibility of a subdomain, few things are easier than just doing option 1 above.

A final note, if you use a 301 or 308 to redirect traffic, clients will usually cache that permanently (as the redirect repsonse names imply). So if you ever change your mind and host something at the hostname originally being redirected, clients that have visited it will keep redirecting traffic until their cache is cleared or they switch browsers (technically they’re a different client at that point).

Sources