Episode 01 · Site Structure

Create the Site Structure First

Start EP01 by recording the competitors ranking on Google, collecting their sitemap URL inventory, and using AI to group those URLs into a proposed site structure. The sheet now produces both the evidence matrix and a clean page tree showing what pages to include.

Manual input
Ranking page + domain

The ranking page shows what type of page is winning. The domain is used for sitemap discovery.

Script output
Inventory · Matrix · Tree

The script preserves raw sitemap URLs, sends AI a strategic sample, groups page types, and builds a recommended page tree.

Quick workflow

Set up the sheet first

1 · Copy the template

Open the main Google Sheet template and make one copy. Use the same copied sheet for the full series.

2 · Run EP01 setup

Paste the Apps Script below and click Run for setupEP01. The script creates the EP01 tabs.

3 · Add competitors

Add the competitor name, ranking page URL, ranking page type, and domain. Then run the EP01 menu actions to collect inventory and analyze the structure.

Apps Script

EP01 setup script

Setup steps

  • Open this EP01 page.
  • Copy the Apps Script code below.
  • Open your copied Google Sheet. Apps Script works only inside a native Google Sheet, not inside Excel .xlsx mode.
  • Go to Extensions > Apps Script.
  • Paste the code into the Apps Script editor.
  • Click Run for the function named setupEP01.
  • Complete the authorization prompt if Google asks for permission.
  • Return to the Google Sheet and refresh the browser tab.
  • Use SEO/GEO Series > EP01: Find Sitemaps.
  • Use SEO/GEO Series > EP01: Extract Sitemap URLs.
  • Use SEO/GEO Series > EP01: Set AI API Key.
  • Use SEO/GEO Series > EP01: Analyze Site Structure with AI.
  • If you already have the matrix, use SEO/GEO Series > EP01: Build Site Tree from Matrix.

What the script creates

EP01 COMPETITORS

EP01 URL INVENTORY

EP01 AI SAMPLE

EP01 SITE STRUCTURE

EP01 SITE TREE

EP01 AI LOG

What to add manually

  • Search position
  • Competitor name
  • Ranking page URL
  • Ranking page type
  • Domain

Copy this Apps Script

Loading EP01 Apps Script...
Tab 1

EP01 COMPETITORS: record what is ranking

Ranking page URL

Add the exact page that is ranking for the keyword, not only the homepage. This helps identify the page type Google is rewarding for that search.

Manual input

Ranking page type

Classify the ranking page as homepage, service page, location page, blog page, comparison/list page, directory page, or another clear type.

Manual review

Domain

Add the competitor domain. The script uses this domain to check /robots.txt, discover sitemap URLs, and collect sitemap URLs.

Manual input

Generated columns

The script tries to fill robots.txt URL, robots status, sitemap URL, sitemap status, and notes. These fields should not be filled manually unless automation fails.

Script output
Technical basics

Why robots.txt and sitemap are used

What is robots.txt?

robots.txt is a public file usually found at domain.com/robots.txt. It gives crawler instructions and often lists the website sitemap.

What is a sitemap?

A sitemap is usually an XML file that lists website URLs. It can also include priority and last modified date when the website provides those fields.

Why use them?

They help collect competitor pages faster. Instead of opening every page manually, the script uses sitemap data to build a URL inventory for AI-assisted grouping.

Tabs 2, 3, and 4

EP01 URL INVENTORY, AI SAMPLE, SITE STRUCTURE, and SITE TREE

URL inventory first

The inventory tab stores the raw sitemap URLs, original paths, language prefix, functional path, priority, last modified date, and source sitemap. This remains the full evidence archive.

Script output

AI sample next

The AI sample tab shows the smaller coverage-based set used for analysis. It keeps important families like services, locations, work, about, contact, and resources without flooding the prompt with every URL.

Cleaner AI input

AI grouping after sample

The site-structure tab shows an AI-generated page-type matrix. Competitors are listed across C1 to C10, and cells contain the exact paths found for each competitor.

Manual review

Site tree final

The site-tree tab converts the matrix into a clean hierarchy: home, top-level pages, grouped service pages, location pages, resources, and utility pages to include or review.

Planning output

Language is metadata

A path like /ar/service/digital-marketing/ is treated as an Arabic service page, not a location page. Location is inferred only from explicit city, country, region, or local-market terms.

Planning step

Not a copy exercise

The goal is not to copy one competitor. The goal is to understand the category baseline and then decide the right page structure for the website being built.

Human decision
Concept document

Why site structure comes first

What is this process?

Most websites are planned by guessing the pages first and thinking about SEO later. EP01 reverses that order. The site structure is planned before the website is built by studying the websites Google is already rewarding for the main keyword.

The process starts with one target keyword. Search it in an incognito Google tab, collect the competitors ranking on the first page, and record the exact page that is ranking.

The ranking page shows the page type Google is surfacing for that keyword. It may be a homepage, service page, location page, blog page, comparison page, directory page, or another type. That pattern helps decide what kind of page should be created.

After the ranking competitors are listed, the page inventory is collected from their websites. This is where robots.txt and sitemaps make the work faster. AI then groups the inventory into a proposed structure that can be reviewed before design or development starts.

Reel

Instagram embed will be added here once the reel URL is shared.

Supporting PDF

Open EP01 PDF document
At a glance

Step 1 · Search the target keyword

Open Google in an incognito tab and search the main keyword the website should rank for. Incognito is used because normal search results can be personalised by search history, location signals, and account activity.

Step 2 · List the competitors

Record 8 to 10 real competitors from the organic search results. Skip ads, featured snippets, and large directories when they are not the type of website being planned.

Step 3 · Collect URL inventory

Use the competitor domain to find sitemap data, then collect the sitemap URLs into the inventory tab. Keep original paths visible, including language-prefixed URLs.

Step 4 · Build the site structure

Run the AI analysis to group the inventory into page types, competitor coverage, recommendations, and reasoning. The final site structure should use these patterns, not copy one website.

How this connects to the sheet

EP01 COMPETITORS is for the search result research. Add the exact ranking page URL and the competitor domain here. The ranking page explains the page type that is currently winning for the keyword.

EP01 URL INVENTORY is for raw sitemap evidence. It preserves the full audit trail, while AI analysis uses a smaller strategic sample plus an excluded URL summary.

EP01 SITE STRUCTURE is for the AI-generated page-type coverage matrix. It shows which competitors have each page type, exact supporting paths, recommendation, and reasoning.

EP01 SITE TREE is the final planning view. It turns the matrix into a page hierarchy showing what pages to include, where they sit, and why they are recommended.

Why this works

This uses live search results instead of assumptions. Google is already showing what it trusts for the keyword, and competitor page patterns reveal the expected structure for that category.

Important note

The goal is not to copy competitor pages. The goal is to understand the baseline, spot useful gaps, and create a clear structure for the website being built.

Limitations

What automation can and cannot do

Robots.txt may not show a sitemap

The script checks https://domain.com/robots.txt. If the sitemap is not listed there, it also tries /sitemap.xml.

Some websites block automated access

If a competitor blocks requests or hides sitemap data, review the website navigation manually and note the important pages.

Sitemaps may be incomplete

Some important pages may not appear in the sitemap. The sitemap is a helper, not the final answer.

AI suggestions need review

AI grouping is used to reduce noise and find patterns, but it is still a planning assistant. Review the evidence paths before deciding the final site structure.