The Technical SEO Checklist for 2026
Forty checks across six layers — including the AI-crawler layer most sites still miss. · By Karan Checker · ~10 min read
Last updated: · Reviewed quarterly
Technical SEO in 2026 spans six layers: crawl access, indexation control, performance, rendering, structured data, and — the new one — AI-crawler readiness. This is the working checklist we run on every audit, published in full. Work top to bottom; each layer assumes the one above it.
Layer 1: Crawl Access
- robots.txt permits everything you want crawled — and explicitly considers
GPTBot,PerplexityBot,ClaudeBot,Google-Extended(blocking AI crawlers is opting out of AI search) - XML sitemap: current, error-free, submitted, containing only canonical 200-status URLs
- No accidental crawl traps: infinite calendars, session parameters, faceted-filter explosions
- Server returns proper status codes; no soft-404s; redirect chains ≤1 hop
- Crawl-budget check on large sites: log-file review of where bots actually spend time
Layer 2: Indexation Control
- Canonical tags: self-referencing on originals, correct on variants, never conflicting with noindex
- Index bloat audit: parameter URLs, tag pages, thin archives out of the index
- Pagination and faceted navigation with deliberate index/noindex rules
- hreflang correct and reciprocal, if international
- One indexable version per resource: protocol, www, trailing-slash all consolidated
Layer 3: Performance (Core Web Vitals)
- LCP < 2.5s on real-user (CrUX) data, not just lab runs
- CLS < 0.1: dimensions on images/embeds, no late-injected banners
- INP < 200ms: long-task audit, third-party script diet
- Images: modern formats (WebP/AVIF), responsive sizes, lazy-loaded below the fold
- Fonts preconnected/preloaded; render-blocking CSS/JS minimized
Layer 4: Rendering
- Critical content and links present in rendered HTML — verified with URL Inspection, not assumed
- No client-side-only navigation invisible to crawlers
- Key content not gated behind interaction (tabs/accordions that never enter the DOM)
- Remember: most AI crawlers execute little or no JavaScript — if it's not in the initial HTML, LLMs may never see it
Layer 5: Structured Data
- Organization schema with
alternateNameandsameAs— your entity's ID card - Page-type schema where genuinely applicable: Article, Service, Product, FAQPage, BreadcrumbList
- Person schema for authors, linked to bio pages
- Everything validated; no markup describing content that isn't visibly on the page
Layer 6: AI Readiness (the 2026 layer)
llms.txtat root: a curated map of your key pages for machine readers- Answer-first structure on commercial pages: 40–60 word liftable openings
- Visible "last updated" dates and a real refresh cadence on pillar content
- Entity consistency check: does asking three AI engines "what is [your brand]?" return the same, correct answer?
Every item here is checkable in an afternoon with Search Console, a crawler, and honesty. Or have us run the 200-point professional version — the technical SEO audit — which ends in fixes, not just findings. We eat our own checklist: this site's robots.txt and llms.txt are live examples.
How to Actually Run This Checklist (The Two-Afternoon Version)
Afternoon one — evidence gathering: crawl the site with Screaming Frog (the free tier covers 500 URLs), export Search Console's indexing and Core Web Vitals reports, pull a CrUX check on your top templates, and fetch robots.txt, sitemap.xml, and llms.txt. Afternoon two — triage: walk the six layers against the evidence, logging every failure with three fields: severity (does it block crawling/indexing, degrade it, or merely offend perfectionism?), effort, and owner. You'll finish with a prioritized register instead of anxiety — which is the entire difference between an audit and a worry list.
The Five Failures We Find Most Often
- Redirect chains from historical migrations — http→https→www→trailing-slash, three hops deep, quietly taxing every crawl and every link's equity.
- Index bloat from parameters and tags — thousands of near-duplicate URLs diluting the crawl while money pages wait days for re-indexing.
- Client-side content invisible to non-Google crawlers — fine in Googlebot's renderer, absent for GPTBot and friends, which is why a site can rank yet never be cited.
- Schema that contradicts itself — one Organization name in the homepage markup, another in the footer, a third on the About page. Entity ambiguity, self-inflicted.
- The 2023 AI-crawler blocklist nobody revisited — a panic-era robots.txt still refusing GPTBot in 2026, silently opting the brand out of AI search.
Severity Triage: What Blocks vs What Degrades
Fix this week: anything preventing crawl or index of commercial pages — robots errors, noindex accidents, 5xx instability, canonical conflicts. Fix this month: Core Web Vitals failures on template level, redirect chains, schema invalidity, rendering gaps. Fix this quarter: index bloat cleanup, internal-linking improvements, llms.txt and answer-structure passes. Sequencing by severity is what separates a useful audit from the 60-page PDF that dies in a drawer — and it's why our professional audits ship as a register with owners and dates, not a document with a cover page.
Keeping It Fixed: The Regression Problem
Technical SEO decays at the speed of your deploy cycle: every release can reintroduce a noindex, break a canonical, or ship a layout shift. The countermeasures are cheap — a pre-launch checklist item in your release process, automated weekly crawls diffed against baseline, and Search Console alerts actually routed to a human. Sites that institutionalize these three stay clean for years; sites that don't re-buy the same audit every eighteen months.
Tooling: The Minimal Stack That Covers All Six Layers
You don't need enterprise spend to run this checklist properly. The working minimum: a desktop crawler (Screaming Frog's free tier for small sites; paid for anything real) covering layers 1–2 and much of 5; Google Search Console — free and non-negotiable — for indexation truth, Core Web Vitals field data, and rendering checks via URL Inspection; PageSpeed Insights / CrUX for layer 3 at template level; the schema validator for layer 5; and your server log files — the most under-used free dataset in SEO — for where crawlers actually spend time, including which AI bots visit and what they fetch. Layer 6 needs no tooling at all beyond honesty: open your robots.txt, check for an llms.txt, and paste your key page into a plain-text extractor to see what a non-rendering crawler receives. The pattern worth internalizing: every layer has a free source of truth, and paid tools mostly add scale, scheduling, and prettier exports. Teams fail this checklist not for lack of budget but for lack of cadence — which is why the last section of this post exists, and why it's the one to actually implement.
Key Takeaways
- Work the six layers in order — crawl access, indexation, performance, rendering, structured data, AI readiness — each assumes the one above it.
- The 2026 layer most sites miss: AI-crawler permissions, llms.txt, answer-first structure, and visible update dates.
- Most AI crawlers execute little or no JavaScript — client-side-only content is invisible to LLM retrieval.
- Technical SEO decays at deploy speed; automated weekly crawls diffed against baseline beat annual heroics.
Rather have professionals run it?
Our 200-checkpoint audit covers all six layers with prioritized fixes and verification. Free mini-scan on a call first.
Book a Free Mini-AuditGot questions? We've got answers.
What is technical SEO?
Technical SEO is the work of making a website fully accessible, understandable, and fast for search engines and crawlers — covering crawlability, indexation, performance, rendering, and structured data. It's the foundation every other SEO effort stands on.
How often should I run a technical SEO audit?
A full audit annually, an event-driven audit after any migration, redesign, or unexplained traffic drop, and lightweight monitoring continuously. Sites that deploy code frequently introduce technical regressions frequently — the audit cadence should match the release cadence.
What are Core Web Vitals thresholds in 2026?
The passing bars remain: Largest Contentful Paint under 2.5 seconds, Cumulative Layout Shift under 0.1, and Interaction to Next Paint under 200 milliseconds — measured at the 75th percentile of real-user data, not lab tests.
Do AI crawlers read JavaScript?
Mostly no — GPTBot, PerplexityBot, and ClaudeBot execute little or no JavaScript, unlike Googlebot's full rendering. Content that only exists after client-side rendering is effectively invisible to LLM retrieval, which is a strong argument for server-rendered critical content.
What is llms.txt and do I need one?
llms.txt is a proposed root-level text file giving LLMs a curated, structured summary of your site's important pages — a sitemap written for machine readers. It's low-effort, zero-risk, and increasingly checked; there's no good reason not to have one.
