Duplicate content: titles, canonical and URL variants

What duplicate content is

Duplicate content means the same or very similar main content at more than one address. Most of it is not copied text but technical: one page answers under several addresses. Google calls the choice of one representative address canonicalization. It groups pages with the same or very similar main content and shows one of them in search results.

Google writes that some duplicate content is normal and not a violation of its spam policies. The problem is a different one: you do not decide which address appears, signals such as links are split across several addresses, statistics are harder to read, and crawlers spend time on copies instead of new pages.

Where duplicates come from

• Protocol and host: http and https, with and without www. Google names both as typical variants.
• Parameters: tracking parameters such as gclid or utm_source, session IDs, and sorting or filters on category pages.
• Small differences in the address: a slash at the end, upper and lower case, or index.html can turn one page into several addresses.
• The same product in several categories, with one address for each path.
• Print views, separate mobile pages, and test or demo copies that crawlers can reach.
• Region variants: separate addresses for several countries with the same content in the same language.

To find them, compare three lists: the addresses in your sitemap, the addresses that appear in your statistics, and the addresses that a crawl of your site finds. An address that shows up in your statistics but is missing from the sitemap is often a variant.

Duplicate titles, descriptions and H1

A second kind of duplicate is about texts, not addresses. Google advises a distinct, descriptive title for every page and warns against repeated or boilerplate text in the title element. Its example is a shop that uses the same sales phrase as the title of every page, so that the pages can no longer be told apart. If the title is not useful, Google can also draw on the main visible title and on headings such as h1.

Typical causes are templates that only fill in the shop name, category pages with page numbers, and product variants that share one text. Give each page in such a group a title, a description and an H1 that name what is different: the product, the colour, the city or the topic. How to write them is covered in our guide on titles and meta descriptions.

Pages that are almost the same

Pages with almost the same main text are harder to judge: location pages that only change the city name, variants with one shared text, tag and category pages that list the same articles. Ask for each group whether each page has its own purpose for visitors. If it has, give it its own content. If not, merge the pages into one and redirect the others, or mark one page as the main version.

How to tell Google which address is the main one

Google names three ways and says how strong each one is as a signal:

• A redirect, for example 301: a strong signal that the target should become canonical. Use it when you retire the duplicate.
• A canonical link in the head of the page or as an HTTP header: also a strong signal. The header works for files such as PDFs too.
• A sitemap that lists only main addresses: a weak signal.

Google treats your choice as a hint, not a rule, and can pick another page if other signals disagree. So let every signal point to the same address.

Rules for canonical links

• Every main page names itself as canonical.
• Use full addresses including https://, not relative ones.
• Do not name different canonical addresses for the same page in different places.
• Do not use noindex to choose between duplicates on your own site. It keeps the page out of Search completely.
• Do not use robots.txt for this. Google may still index blocked addresses, only without their content.
• Link internally to the main address, in menus, in the text and in the sitemap.
• Language versions name themselves, not another language. Our guide on hreflang mistakes explains why.

www, https and other site variants

Decide on one form of your domain, for example https with www, and redirect every other form to it, page by page and not to the home page. Google generally prefers the https version as canonical unless there are problems such as an invalid certificate, and it advises against redirects from https to http. Afterwards, check that internal links, canonical links and the sitemap all use the chosen form.

Work with Do4U

The Duplicate content check reads your website from the home page, from a list of addresses or from a sitemap. For a limited number of pages it compares titles, meta descriptions, H1, canonical links, noindex and the visible main text. You get each finding grouped with the affected addresses: duplicate and missing titles and descriptions, pages with little text, pages with almost the same text, canonical links to broken pages or other websites, and noindex pages that your own pages link to. You can export everything as CSV. The check runs without AI and is free, with a daily limit per account.

Pages are read as the server delivers them, without JavaScript, and a large website is read only in part. Title lengths are guide values, not rules of Google. Do4U finds hints, you check and decide which address stays.