On June 18, Tobi Lutke – Shopi­fy CEO, announced on Twit­ter that Shopi­fy stores are now able to edit their robots.txt file which gives store own­ers more con­trol over how search engines crawl their site.

The Default Robots.txt File in Shopify

All Shopi­fy stores include a default robots.txt file that is opti­mal for Search Engine Opti­mi­sa­tion (SEO) and Shopi­fy notes that this works for most stores, but you can now edit the file as described in the steps below to:

The robots.txt file is locat­ed at the root direc­to­ry of your Shopi­fy store’s pri­ma­ry domain name (this is also the case for the major­i­ty of web­sites). For exam­ple: johns​-appar​el​.com/​r​o​b​o​t​s.txt.

Fol­low­ing the steps described below to edit this file, you will find the fol­low­ing default code used by Shopi­fy to gen­er­ate the robots.txt file. This incor­po­rates Shopi­fy’s default rules and by leav­ing this code in place, you will ensure that if Shopi­fy updates their own rules — they will auto­mat­i­cal­ly apply to your robots.txt file.

# we use Shopify as our ecommerce platform
{%- comment -%}
# Caution! Please read https://help.shopify.com/en/manual/promoting-marketing/seo/editing-robots-txt before proceeding to make changes to this file.
{% endcomment %}
{% for group in robots.default_groups %}
{{- group.user_agent -}}

{% for rule in group.rules %}
{{- rule -}}
{% endfor %}

{%- if group.sitemap != blank -%}
{{ group.sitemap }}
{%- endif -%}
{% endfor %}

Edit­ing The Robots.txt File in Shopify

It is impor­tant to note that while Shopi­fy pro­vides the abil­i­ty to edit this file, Shopi­fy Sup­port can’t help you with edits to this file. Incor­rect use of this file could result in loss of traf­fic or your store no longer being indexed by search engines.

If you are unsure, Shopi­fy Part­ners like us can help you to edit this code.

Steps:

The fol­low­ing steps cre­ate a robots.txt.liquid file (rather than a robots.txt file that sites might tra­di­tion­al­ly use).

  1. From your Shopi­fy admin, go to Online Store > Themes.
  2. Click Actions, and then click Edit Code.
  3. Click Add a new tem­plate, and then select robots.txt.
  4. Click Cre­ate tem­plate.
  5. Make the changes that you want to make to the default tem­plate. For more infor­ma­tion on liq­uid vari­ables and com­mon use cas­es, refer to Shopi­fy’s Devel­op­er page Cus­tomize robots.txt.liquid.
  6. Save changes to the robots.txt.liquid file in your pub­lished theme.

Ver­sion Control

It is also impor­tant to note that if your Shopi­fy store theme is cur­rent­ly being ver­sion-con­trolled and deployed using Shopi­fy’s The­meK­it, com­mand line changes or GitHub, then you should create/​make the changes to your robots.txt.liquid file through these processes.

To find out more, read Shopi­fy’s Edit­ing robots.txt.liquid help article.