Quick answer: A technical SEO audit checks whether important pages return the right status, can be crawled, render useful content, use intentional index and canonical directives, appear in sitemaps, work on mobile, load securely and expose valid structured data. It should also verify AI-search crawler access and monitor server logs, Search Console and Bing Webmaster Tools.
Technical SEO creates the conditions for discovery. It cannot compensate for weak or duplicate content.
Technical SEO checklist
- Important pages return HTTP 200
- Internal links are crawlable
- Robots.txt is intentional
- Noindex directives are reviewed
- Canonicals point to preferred URLs
- Redirects are direct and relevant
- XML sitemaps contain canonical indexable URLs
- Mobile content matches desktop
- JavaScript renders essential content
- HTTPS and security headers work
- Core Web Vitals are monitored
- Structured data validates
- Orphan pages are resolved
- OAI-SearchBot policy is deliberate
- IndexNow and Bing diagnostics are configured where useful
1. Check status codes
Important indexable pages should normally return 200 OK.
Review:
- 3xx redirects;
- 404 and 410 removals;
- soft 404 pages;
- 5xx server errors;
- blocked assets;
- timeout patterns.
Use 301 or 308 for intended permanent moves and 302 or 307 for genuinely temporary redirects. Keep chains short and map old pages to the most relevant new destination.
2. Verify crawlable internal links
Use standard anchors with destination URLs. Important pages should not rely only on:
- form submissions;
- JavaScript click handlers;
- internal search results;
- login-only navigation;
- dynamically generated URLs with no links.
Run an internal crawl and find orphan pages. Use internal linking for SEO to prioritize hubs and high-value pages.
3. Audit robots.txt
Robots.txt controls crawler access. Verify that it does not block:
- important routes;
- essential CSS or JavaScript;
- canonical content;
- product and blog directories;
- sitemaps.
Do not use robots.txt as a privacy control. Blocked URLs can still be known through links, and crawlers cannot see a page-level noindex when they cannot fetch the page.
4. Audit noindex directives
Use noindex for pages that should not appear in supported search indexes, such as certain internal dashboards or thin utility routes.
Find accidental noindex on:
- product pages;
- category pages;
- blog posts;
- paginated hubs;
- newly migrated content.
Review both HTML meta tags and X-Robots-Tag headers.
5. Validate canonical URLs
A canonical signals the preferred version among duplicates. It should:
- point to an indexable URL;
- use the preferred protocol and hostname;
- match redirects and internal links;
- avoid chains;
- remain consistent with sitemap entries;
- be self-referential on unique canonical pages where appropriate.
Canonical is a hint, not an absolute command. Contradictory signals can lead search engines to choose another URL.
6. Review XML sitemaps
Include canonical, indexable and valuable URLs. Exclude:
- redirects;
- 404 pages;
- noindex pages;
- duplicate parameter URLs;
- private routes;
- alternate URLs canonicalized elsewhere.
Add accurate lastmod only when the page materially changes. Submit the sitemap in Google Search Console and Bing Webmaster Tools.
7. Test JavaScript rendering
Compare raw HTML and rendered output. Essential content, links and metadata should be reliably available.
Check:
- server-side rendering or static output;
- hydration errors;
- client-only canonical tags;
- content hidden behind interaction;
- API failures;
- infinite scrolling without paginated URLs;
- blocked scripts.
Use URL Inspection to see how Google processed the page.
8. Verify mobile parity
Google uses mobile-first indexing. Mobile pages should have the same primary content, headings, structured data, alternative text and canonical intent as desktop.
Test:
- responsive layout;
- readable typography;
- tap targets;
- navigation;
- image dimensions;
- overlays;
- performance on slower devices.
9. Monitor Core Web Vitals
Core Web Vitals currently focus on:
- Largest Contentful Paint (LCP);
- Interaction to Next Paint (INP);
- Cumulative Layout Shift (CLS).
Use field data when available. Lab tools help diagnose but do not replace real-user measurement.
Performance is part of page experience; it is not a substitute for relevance and helpful content.
10. Secure the site with HTTPS
Confirm:
- valid certificate;
- automatic HTTP-to-HTTPS redirect;
- no mixed content;
- secure cookies;
- consistent canonical HTTPS URLs;
- no duplicate HTTP indexation;
- safe external scripts.
Security failures can block users and crawlers.
11. Validate structured data
Use schema types supported for the visible content. Check required fields and ensure markup matches what users can see.
For IndieTools, possible types can include:
Articlefor blog posts;SoftwareApplicationfor eligible product pages;BreadcrumbListfor navigation;OrganizationorPersonwhere accurate.
Structured data does not guarantee a rich result or AI citation.
12. Control duplicate routes and parameters
Audit:
- trailing slash variants;
- uppercase and lowercase paths;
- tracking parameters;
- filters and sorting;
- legacy category-first product routes;
- print or preview URLs;
- HTTP and hostname variants.
Use consistent internal links, redirects and canonicals. Avoid creating crawlable combinations with no unique value.
13. Check AI-search crawler access
Google says no special file is required for AI Overviews or AI Mode; normal Google Search eligibility applies.
For ChatGPT search, OpenAI advises publishers not to block OAI-SearchBot if they want content discoverable and cited. Decide this policy deliberately in robots.txt and security controls.
Read how to optimize for AI search.
14. Use IndexNow where appropriate
IndexNow lets participating search engines know when URLs are added, updated or deleted. Microsoft recommends it alongside sitemaps for freshness in AI-powered search.
IndexNow is a notification mechanism, not a ranking or indexing guarantee. Send only changed canonical URLs and monitor errors.
15. Monitor the right evidence
Use:
- server access logs;
- Google Search Console;
- Bing Webmaster Tools;
- URL inspection;
- sitemap reports;
- Core Web Vitals field data;
- structured-data reports;
- application uptime monitoring;
- crawl comparisons before and after releases.
IndieTools-listed Preflight.sh, SEOReport and TheFastestWeb relate to audit or monitoring workflows; verify current product details.
Audit priority matrix
| Severity | Example | Action |
|---|---|---|
| Critical | Sitewide noindex or 5xx | Fix immediately |
| High | Canonicals point to wrong domain | Fix before recrawl spreads |
| Medium | Orphan important pages | Add relevant internal paths |
| Low | Minor metadata duplication | Schedule cleanup |
| Monitor | Small lab-performance variance | Confirm with field data |
Frequently asked questions
Does technical SEO guarantee indexing?
No. It removes barriers and clarifies intent, but search engines decide what to index.
Do I need llms.txt for Google AI search?
Google says it does not use llms.txt or special AI text files for its Search generative features.
Is an XML sitemap required?
Not always, but it is strongly useful for large, new, changing or poorly linked sites.
How often should I run a technical audit?
Continuously monitor critical signals and run deeper audits after releases, migrations or major architecture changes.
