catalinaBlue

Blog

Web Development

What is your budget?

Consent to Hanabi collecting and storing data*

Let's get to work. If you are thinking about launching or updating your e-commerce store and you think we'd be a good fit, then we would love to hear from you. Fill out our project form below to discuss how we can help your business and we'll get back to you in 24 hours.

Get your free consultation

Grow Your Business Online With HANABI

Tell us about the project*

What services you are interested in?*

Work with us

On June 18, Tobi Lutke – Shopify CEO, announced on Twitter that Shopify stores are now able to edit their robots.txt file which gives store owners more control over how search engines crawl their site.

All Shopify stores include a default robots.txt file that is optimal for 

 and Shopify notes that this works for most stores, but you can now edit the file as described in the steps below to:

allow or disallow certain URLs from being crawled

add crawl-delay rules for certain crawlers

 file is located at the root directory of your Shopify store's primary domain name (this is also the case for the majority of websites). For example 

Following the steps described below to edit this file, you will find the following default code used by Shopify to generate the robots.txt file. This incorporates Shopify's default rules and by leaving this code in place, you will ensure that if Shopify updates their own rules - they will automatically apply to your robots.txt file.

# we use Shopify as our ecommerce platform
{%- comment -%}
# Caution! Please read https://help.shopify.com/en/manual/promoting-marketing/seo/editing-robots-txt before proceeding to make changes to this file.
{% endcomment %}
{% for group in robots.default_groups %}
 {{- group.user_agent -}}

 {% for rule in group.rules %}
 {{- rule -}}
 {% endfor %}

 {%- if group.sitemap != blank -%}
 {{ group.sitemap }}
 {%- endif -%}
{% endfor %}

It is impor­tant to note that while Shopi­fy pro­vides the abil­i­ty to edit this file, Shopi­fy Sup­port can’t help you with edits to this file. Incor­rect use of this file could result in loss of traf­fic or your store no longer being indexed by search engines.
If you are unsure, Shopi­fy Part­ners like us can help you to edit this code.

The fol­low­ing steps cre­ate a robots.txt.liquid file (rather than a robots.txt file that sites might tra­di­tion­al­ly use).

Make the changes that you want to make to the default tem­plate. For more infor­ma­tion on liq­uid vari­ables and com­mon use cas­es, refer to Shopi­fy’s Devel­op­er page 

Save changes to the robots.txt.liquid file in your pub­lished theme.

It is also impor­tant to note that if your Shopi­fy store theme is cur­rent­ly being 

 and deployed using Shopi­fy’s The­meK­it, com­mand line changes or GitHub, then you should create/​make the changes to your robots.txt.liquid file through these processes.

Shopify has released a new feature to enable stores to edit their robots.txt file, enabling greater control for a Shopify store's SEO.

Shopify Stores Can Now Edit Their Robots.txt File

Hanabi is a strategic digital agency, helping clients not just build websites, but create brands.

Work with us

Grow Your Business Online With HANABI

Get your free consultation

Web Development

Sign up to our newsletter for the latest digital marketing and e-commerce news.