Every directory we run at Kings Hospitality Group eventually crosses the line where one sitemap file stops being useful. A new venue site might launch with a few hundred listings and a handful of guides. Within a year it can hold tens of thousands of pages across listings, categories, locations, and an expanding knowledge hub. At that scale a single sitemap is a blunt instrument. Splitting it into themed files is the difference between guessing at your indexation and actually reading it.
Why one big sitemap fails you at scale
A monolithic sitemap technically works. Search engines will fetch it, parse it, and use it. The problem is not whether it functions, the problem is what it tells you afterward. When forty thousand URLs sit in one file and your coverage report says thirty one thousand are indexed, you have learned almost nothing actionable. Which nine thousand fell out? Listings? A category that broke? A batch of thin location pages? You cannot tell, because the unit you submitted does not match the unit you think in.
The hard limits are real and worth stating plainly. One sitemap file may contain up to fifty thousand URLs and may be no larger than fifty megabytes uncompressed. Cross either and the file is invalid, and a quietly invalid sitemap is one of the slower failures to notice because nothing visibly breaks. But we never get close to those ceilings on purpose, and the reason is diagnostic rather than technical.
Split by what you can act on
The right way to split is by the sections you would investigate separately. On a typical directory that means a sitemap per content type. Listings in one, category and taxonomy pages in another, location pages in a third, and the knowledge or editorial content in its own file. If a property is large enough, we split listings further by region or by first letter so no single file carries the whole catalogue.
The test is simple. Ask yourself: if indexation dropped here, would I treat this group of pages as one problem? If yes, they belong in one sitemap. Editorial guides and machine generated listing pages have completely different risk profiles, so they never share a file. This mirrors how we think about the whole crawl surface, which is why a flat architecture that keeps important pages shallow pairs so well with sectioned sitemaps. The structure of the site and the structure of the sitemap should tell the same story.
A sensible file structure
Here is the shape we use across the portfolio. The exact names matter less than the discipline of keeping them stable.
- sitemap.xml as the index, the single file you submit.
- sitemap-listings-1.xml, sitemap-listings-2.xml and so on, chunked so each holds a few thousand entries.
- sitemap-categories.xml for taxonomy and hub pages.
- sitemap-locations.xml for geographic pages.
- sitemap-knowledge.xml for articles and guides.
Each child sitemap is listed in the index with its own last modified date. That date is not decoration. It is how a crawler decides which files are worth refetching, so it must reflect genuine change inside that section and nothing else.
The sitemap index is the control panel
The sitemap index is a small XML file that does one job: it lists every child sitemap and when each last changed. You submit only the index. From then on the crawler walks down to the children on its own schedule. This is what makes the whole arrangement maintainable, because adding a new section is one new line in the index rather than a restructure.
Keep the index honest. If a child sitemap has not changed, do not bump its last modified date. Crawlers learn the rhythm of a site partly from how truthfully these dates move, and a sitemap that claims everything changed every night trains them to ignore the field entirely. We treat the index as a promise about freshness, which connects directly to how we handle last reviewed dates and genuine freshness signals on the pages themselves.
Reading indexation as a per section signal
This is the entire payoff. When sitemaps are split by section, your coverage report becomes a dashboard. Submit each child file and you can watch the submitted versus indexed ratio move independently for listings, categories, locations, and editorial. A healthy editorial section sitting at near full indexation while listings sag tells you exactly where to look, and usually points at thin or duplicative listing pages rather than a sitewide fault.
We review these ratios weekly. A section that drifts down two or three weeks running gets pulled apart before it becomes a habit. Often the cause is mundane: a template change introduced a soft duplicate, or a batch of new listings shipped without enough unique content to earn a place. Catching that early is only possible because the sitemap boundary matches the editorial boundary. If you want the broader picture of how new pages earn their place in the index, our guide on getting a new site indexed quickly covers the launch side of the same discipline.
What to put in, and what to leave out
A sitemap is a list of the canonical pages you want indexed. It is not a dump of every URL the server can produce. Leave out anything that is noindexed, anything that canonicalises elsewhere, anything behind a parameter, and anything returning a non two hundred status. A sitemap full of redirects and dead URLs erodes trust in the file, and that trust is the only thing the file has to trade on.
Practically, that means the sitemap should be generated from the same source of truth that decides what is canonical, not from a raw crawl. When those two disagree, the sitemap becomes noise, and noisy sitemaps are worse than no sitemap because they actively mislead your coverage analysis.
Automate generation, never edit by hand
At a few thousand URLs you might be tempted to hand maintain a file. Do not. Sitemaps should regenerate automatically whenever content publishes or changes, with the last modified dates driven by real edit timestamps. We rebuild ours on publish and on a nightly sweep, so the index always reflects the live state of each section without anyone touching XML.
The automation also enforces the rules: it excludes noindexed and non canonical URLs, it chunks files before they grow unwieldy, and it updates the index. Human edited sitemaps drift out of sync within weeks, and a stale sitemap quietly teaches crawlers that your freshness signals cannot be trusted.
How this fits the wider build
Sectioned sitemaps are one piece of how we keep large directories crawlable and legible, and the approach is the same whether a site has five thousand pages or five hundred thousand. It is a small part of the broader operating method we describe in our building thesis, where the recurring theme is that good infrastructure makes problems visible early. A sitemap that mirrors your site sections turns a vague worry about indexation into a precise, weekly readable signal, and that is worth far more than the convenience of a single file.
Start simple. Split by content type, cap each file well below the limits, point a clean index at them, automate the whole thing, and then actually read the coverage data section by section. The first time a single sagging section tells you exactly where a template broke, the discipline pays for itself.
Our internal rule at Kings Hospitality Group is the five thousand URL ceiling per sitemap file, well under the fifty thousand limit, so a single dropped section is obvious in coverage reports rather than buried.
Common questions
How many URLs can one sitemap hold?
A single XML sitemap can list up to fifty thousand URLs or fifty megabytes uncompressed. We cap ours far lower so each file maps to one readable section.
Do I need a sitemap index?
Yes, once you have more than one sitemap file. The index is a small file that lists every child sitemap, and it is the single URL you submit to search engines.
Should sitemaps be compressed?
Gzip is fine and search engines accept it, but the fifty megabyte limit applies to the uncompressed size, so plan around the raw file.