SEO & AI Search

The Technical SEO Checklist for Launching a New Website

Everything that has to be right under the hood before a new website goes live, from robots.txt and canonical URLs to redirects, structured data, Core Web Vitals and the first 30 days after launch.

10 min readHiddlins Editorial Team

A new website gets one first impression with search engines. If the launch ships with a leftover noindex tag, a staging domain in the canonical URLs or a missing redirect map from the old site, you can lose weeks of visibility while crawlers sort out the mess. This technical SEO checklist covers what we verify on every launch: crawling, indexing, canonical URLs, redirects, structured data, performance, security and the first 30 days after go-live. Each item is practical and testable, and none of it depends on tricks.

Why technical SEO comes first#

Content and links decide how well you rank. Technical SEO decides whether you can rank at all. A search engine has to discover a URL, crawl it, render it, decide whether it is worth indexing, and then pick a canonical version among any duplicates. A failure at any of those steps hides your best content, however good it is.

New sites are especially exposed for three reasons:

  • No history. Search engines have no prior signals about your domain, so they crawl cautiously, and a confusing structure wastes the attention they do give.
  • Development leftovers. Staging settings like “Discourage search engines” in WordPress, hard-coded test URLs and placeholder content often survive into production.
  • Redesign migrations. If the new site replaces an old one, every URL that changes without a redirect throws away the links and rankings the old page earned.

Google’s own SEO Starter Guide is a good baseline. The checklist below turns those principles into concrete launch tasks.

Protect staging, then plan the switch#

Most launch-day disasters begin on the staging server. Your development environment has to stay invisible while you build, and every one of those protections has to come off cleanly when you go live.

Use authentication, not just robots.txt#

A Disallow: / rule in robots.txt asks crawlers not to fetch pages, but it does not stop a URL from being indexed if someone links to it. The URL can still appear in results without a description. The reliable option is HTTP authentication on the whole staging host, with a noindex response header as a second layer:

# Staging server only: password-protect and noindex everything
server {
    server_name staging.example.com;

    auth_basic           "Staging";
    auth_basic_user_file /etc/nginx/.htpasswd;

    add_header X-Robots-Tag "noindex, nofollow" always;
}

Write a launch-switch list#

Write down every environment-specific setting before launch day, so nobody is relying on memory at 11 p.m. Typical entries include:

  • WordPress Settings → Reading → Discourage search engines unchecked on production.
  • X-Robots-Tag headers and <meta name="robots" content="noindex"> removed.
  • Site URL, home URL and any hard-coded staging domains replaced in the database (use a serialisation-safe search-and-replace tool for WordPress).
  • Analytics and tag manager IDs switched from test to production properties.
  • Transactional email and payment gateways moved out of sandbox mode.

Crawlability: robots.txt, status codes and internal links#

A minimal, correct robots.txt#

Most small and medium sites need a short robots.txt. Its job is to keep crawlers out of low-value areas (internal search results, cart and account pages) and to point to your sitemap. It is not a security tool. Anything you list is publicly readable. Google’s robots.txt introduction explains the syntax in detail.

User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Disallow: /?s=
Disallow: /search/
Disallow: /cart/
Disallow: /checkout/
Disallow: /my-account/

Sitemap: https://www.example.com/sitemap_index.xml
Do not block CSS or JavaScript

  • Google renders pages much as a browser does. If robots.txt blocks your theme’s stylesheets or scripts, Google may see a broken layout and misjudge the page, including how it looks on mobile.
  • Do not use robots.txt to remove pages from the index. A page blocked from crawling can’t show Google its noindex tag.

Status codes that tell the truth#

Every URL should return the status code that matches what it actually is:

  • 200 for real, indexable content.
  • 301 (or 308) for permanent moves. Use 302/307 only for genuinely temporary situations.
  • 404 or 410 for content that does not exist. A “page not found” message served with a 200 status is a soft 404, and it wastes crawl attention.
  • 503 with a Retry-After header during planned maintenance, so crawlers come back later instead of recording errors.

Internal linking and crawl depth#

Crawlers find pages by following links. Every important page should be reachable through normal <a href> links. JavaScript click handlers don’t count. Keep key pages within about three clicks of the homepage. Orphan pages, which appear in the sitemap but have no internal links, are a common finding in audits. They signal that the site’s own navigation doesn’t consider them important. Clean, semantic HTML with real navigation landmarks helps crawlers and assistive technology alike.

Indexability and canonical URLs#

Pick one canonical host#

Your site is reachable at four or more variants unless you consolidate them: http:// and https://, with and without www, sometimes with and without trailing slashes. Pick one form and 301-redirect every other variant to it in a single hop:

# Redirect all HTTP and non-www traffic to https://www.example.com
server {
    listen 80;
    server_name example.com www.example.com;
    return 301 https://www.example.com$request_uri;
}
server {
    listen 443 ssl;
    server_name example.com;
    # ssl_certificate ... (certificate must cover the bare domain too)
    return 301 https://www.example.com$request_uri;
}

Self-referencing canonical tags#

Every indexable page should declare its preferred URL with an absolute, self-referencing canonical tag. This protects you when the same content is reachable under tracking parameters, sort orders or session IDs:

<link rel="canonical" href="https://www.example.com/services/web-design/">

A canonical tag is a strong hint, not a command. Google weighs it against redirects, internal links and sitemap entries, so all of those signals should agree. Google’s guide to specifying canonical URLs covers the edge cases. The most common launch mistake is canonical tags that still point at the staging domain. Crawl the live site and check for this specifically.

Thin and duplicate pages#

Content management systems generate pages you never asked for: tag archives with a single post, author archives on a single-author blog, attachment pages for every uploaded image, and filtered product listings. Decide deliberately which of these deserve to be indexed. In WordPress, attachment pages should redirect to the file or parent post. Single-post tag archives are usually better as noindex, follow until they grow.

URL structure and redirect mapping#

Good URLs are short, lowercase, hyphen-separated, readable and stable. Decide the structure during planning, because changing it after launch means another round of redirects. Avoid dates in evergreen content URLs, file extensions you might drop later, and deep folder hierarchies that mirror your internal org chart instead of how users think.

The redirect map for a redesign#

If the new site replaces an existing one, the redirect map is the single most important SEO deliverable of the project. The process:

  1. Inventory the old URLs. Combine a full crawl, the old XML sitemap, analytics landing pages and Search Console’s top pages. URLs with backlinks matter most.
  2. Map each old URL to its closest new equivalent. Map one to one where you can. Don’t redirect everything to the homepage. Search engines tend to treat mass homepage redirects as soft 404s.
  3. Implement the redirects as single-hop 301s at the server or with a redirect plugin. Avoid chains like old → interim → new.
  4. Test before DNS changes, using a script that requests every old URL and checks for one 301 that lands on a 200.
  5. Keep the redirects in place long term. Google’s site-move guidance recommends keeping them for at least a year, and in practice there is rarely a reason to remove them at all.
# Quick redirect check: each line of old-urls.txt is a full URL
while read -r url; do
  result=$(curl -s -o /dev/null -w "%{http_code} %{num_redirects} %{url_effective}" -L "$url")
  echo "$url -> $result"
done < old-urls.txt

Any line that shows more than one redirect, or a final status other than 200, needs fixing before launch.

On-page technical elements#

Titles, descriptions and headings#

  • Unique <title> on every page, leading with the page’s subject. Search engines may rewrite titles that are vague, stuffed with keywords or duplicated.
  • Meta descriptions don’t directly affect rankings, but they are often used as the snippet. Write them as a short pitch for the page.
  • One clear <h1> per page and a logical <h2>/<h3> hierarchy. Headings describe structure, so don’t use them for styling.
  • lang attribute on the <html> element, and hreflang annotations if you publish multiple language or regional versions.

Structured data#

Structured data (JSON-LD using the schema.org vocabulary) helps search engines understand what a page represents and makes it eligible for certain rich results. For a typical business site, start with Organization or LocalBusiness on the homepage, BreadcrumbList sitewide, Article on blog posts and Product on shop pages. The markup must describe content that is actually visible on the page.

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "Organization",
  "name": "Example Studio",
  "url": "https://www.example.com/",
  "logo": "https://www.example.com/logo.png",
  "sameAs": [
    "https://www.linkedin.com/company/example-studio",
    "https://github.com/example-studio"
  ]
}
</script>

Validate markup with Google’s Rich Results Test and the Schema Markup Validator before launch. Structured data also helps AI-driven search experiences identify your brand, which we cover in our guide to generative engine optimization.

XML sitemaps#

Your sitemap should list only canonical, indexable URLs that return 200: no redirects, no noindex pages, no parameter variants. Google’s sitemap documentation sets the limits at 50,000 URLs or 50 MB uncompressed per file. It also states that <priority> and <changefreq> are ignored, and that <lastmod> is used only when it is consistently accurate. Let your CMS generate the sitemap, then audit what it includes.

Performance, mobile and security#

Core Web Vitals#

Page experience isn’t a magic ranking lever, but slow, unstable pages lose visitors whatever their position. Google assesses Core Web Vitals at the 75th percentile of real-user visits, with these “good” thresholds:

Metric Measures Good Poor Common launch culprits
LCP (Largest Contentful Paint) Loading of the main content ≤ 2.5 s > 4.0 s Unoptimised hero images, slow server response, render-blocking CSS
INP (Interaction to Next Paint) Responsiveness to input ≤ 200 ms > 500 ms Heavy third-party scripts, large JavaScript bundles, long tasks
CLS (Cumulative Layout Shift) Visual stability ≤ 0.1 > 0.25 Images without dimensions, late-loading banners, web font swaps

A brand-new site has no field data yet, so use lab tools (Lighthouse, WebPageTest) on representative templates before launch: the homepage, a service page, a blog post and a product page. Then watch the field data as it builds up. Our Core Web Vitals guide goes deeper on fixing each metric. Server response time also depends heavily on your platform, which we compare in choosing web hosting.

Mobile-first indexing#

Google primarily crawls and indexes with a smartphone user agent. The mobile version of a page is the version that gets indexed. Make sure content, structured data, internal links and meta tags are identical across breakpoints. Don’t hide text in mobile layouts that you want to rank for.

HTTPS and security headers#

  • A valid certificate covering every hostname you serve, with automatic renewal.
  • No mixed content. Every image, script and font loads over HTTPS.
  • A Strict-Transport-Security (HSTS) header once you are sure HTTPS works everywhere.
  • Sensible defaults for X-Content-Type-Options, Referrer-Policy and a Content Security Policy where practical.

The launch-day checklist#

Use this table on the day the site goes live. Every row should be verified on the production domain, not on staging.

Check How to verify Pass condition
Indexing allowed View source and response headers on key templates No noindex meta tag or X-Robots-Tag header
robots.txt Open /robots.txt No blanket Disallow: /, sitemap URL listed
Canonical host Request http, non-www and trailing-slash variants Single 301 to the preferred URL
Canonical tags Crawl the site Absolute, self-referencing, production domain only
Redirect map Run the redirect script against old URLs One hop, final status 200
Sitemap Crawl the sitemap URLs All 200, all indexable, all canonical
Structured data Rich Results Test on each template No errors; markup matches visible content
Performance Lighthouse on mobile for each template LCP, CLS and total blocking time within budget
HTTPS Browser console and a crawl No mixed content or certificate warnings
Custom 404 Request a made-up URL Helpful page served with a 404 status

The first 30 days after launch#

Set up Search Console and Bing Webmaster Tools#

Verify the domain property in Google Search Console (DNS verification covers every protocol and subdomain), submit the sitemap, and do the same in Bing Webmaster Tools. Bing’s index also feeds several other search and AI products, so it is worth the ten minutes. Use the URL Inspection tool on your most important pages to request indexing and confirm that Google sees the rendered content you expect.

Watch the reports, not the rankings#

In the first weeks, rankings move around too much to tell you much. These signals are more useful:

  • Page indexing report: look for “Crawled – currently not indexed”, “Duplicate without user-selected canonical” and unexpected exclusions.
  • Crawl stats and server logs: confirm that Googlebot gets fast 200 responses, not timeouts or 5xx errors.
  • 404s from old URLs: every one is a missed redirect. Add it to the map.
  • Core Web Vitals report: fills in once enough real-user data exists.
Key takeaways

  • Protect staging with authentication, and keep a written list of every setting that must change at launch.
  • Consolidate to one canonical host, use self-referencing canonical tags, and keep your sitemap limited to clean 200 URLs.
  • For redesigns, the one-to-one redirect map matters more than anything else on this list.
  • Verify everything on production on launch day, then monitor Search Console’s indexing reports for the first month.

Launch once, launch clean#

Technical SEO isn’t glamorous, and done well it is invisible: pages get found, the right URLs get indexed, and your content competes on its merits. Most of the items above are cheap to get right during the build and expensive to fix after launch, which is why we fold them into the plan from day one. If you are still at that stage, our guide to planning a website project shows where SEO fits into scope and timeline.

Launching soon, or worried that a recent launch or redesign went wrong? Our Performance & Technical SEO Audit works through this checklist and more, and gives you a prioritised list of fixes in plain language. Get in touch and tell us about your site, and we’ll tell you honestly what it needs.

Frequently asked questions

When should technical SEO start on a new website project?

During planning, not the week before launch. URL structure, the CMS, hosting and the redirect map from any old site all affect SEO and are expensive to change later. Treat the checklist as part of the build, then verify it again on launch day.

Should I block my staging site with robots.txt?

Robots.txt only asks crawlers not to fetch pages; it does not reliably keep URLs out of the index if they are linked from elsewhere. Password protection (HTTP authentication) is the safest way to keep a staging site private, with a noindex header as a second layer.

How long does it take for a new website to appear in Google?

There is no fixed timeline. A new site with a submitted sitemap and a few links pointing to it is often discovered within days, but indexing every page and building rankings takes considerably longer. Search Console's URL Inspection tool shows you exactly what Google knows about any URL.

Do sitemap priority and changefreq values matter?

Not for Google. Google's documentation states that it ignores priority and changefreq, and it uses lastmod only when the dates are consistently accurate. Spend your effort on keeping the sitemap clean: only canonical, indexable URLs that return a 200 status.

Written by the Hiddlins Editorial Team

We are a full-stack web development studio. Every article is researched, tested and reviewed according to our editorial policy. Questions or corrections? Tell us.

Have a project in mind?

Tell us what you are building. We reply within one business day with honest advice, a realistic timeline and a fixed-price proposal.