Create & Configure Robots.txt in Ciriks Website Builder

Optimizing your website's search engine visibility starts with managing its indexing settings correctly. One crucial file for this purpose is the robots.txt file, which guides search engines on which pages to crawl or ignore.

This guide walks you through the step-by-step process of creating and configuring a robots.txt file using Ciriks Site Builder.

Step 1: Accessing the Static Pages Section

To begin, follow these steps to navigate to the Static Pages section in your Ciriks Site Builder dashboard:

• Log in to your Ciriks Site Builder account.
• Go to Content Management and click on Static Pages.

Step 2: Creating a New Static Page

Now, let’s create a new static page that will host the robots.txt file:

• Click on Add New Page to open the page creation panel.

Step 3: Configuring the Robots.txt Page

In the robots.txt page settings, configure the following details:

• Page Name: Enter "robots.txt" in the title field.
• URL Format: Ensure the URL follows this structure:

  https://yourdomain.com/robots.txt

• Adding Rules: Insert the robots.txt rules in the HTML section.

Important: Do not add anything in the CSS and JavaScript sections.

Here’s an example of basic robots.txt rules:

User-agent: Googlebot  
Disallow: /nogooglebot/  

User-agent: *  
Allow: /  

Sitemap: https://www.example.com/sitemap.xml

For more details on robots.txt rules, you can check out Google's official guidelines.

Step 4: Saving the Page

Once everything is set up, click Save to store your robots.txt page settings.

Final Thoughts

A well-configured robots.txt file helps search engines understand which pages to crawl and which to exclude, improving your site's SEO. With Ciriks Site Builder, managing this file is simple, allowing you to take full control over your website’s indexing preferences.