Ask a Question

What is robots.txt ?

0 votes
276 views
in Discuss by
edited
What is robots.txt ?

1 Answer

0 votes
by
selected
 
Best answer
robots.txt is a text file that instructs web robots (search engine robots) about how to crawl pages on their website. It is used to manage crawler traffic to the website.

Related questions

0 votes
1 answer 348 views
Which is better — Meta Robot tags or robots.txt?
asked Aug 10, 2022 in Discuss by anonymous
0 votes
1 answer 567 views
What is the extension of java code files? a) .js b) .txt c) .class d) .java
asked Jul 20, 2022 in JAVA by Doubtly (98.9k points)
0 votes
1 answer 452 views
what is Tailwind CSS ?
asked Sep 13, 2022 in Discuss by Doubtly (98.9k points)
0 votes
1 answer 311 views
What is GNU? General Public License
asked Aug 27, 2022 in Discuss by Doubtly (98.9k points)
0 votes
1 answer 397 views
What is the difference between free and paid WordPress theme?
asked Aug 27, 2022 in Discuss by Doubtly (98.9k points)
0 votes
1 answer 365 views
What is the difference between custom themes and normal themes?
asked Aug 27, 2022 in Discuss by Doubtly (98.9k points)

Doubtly is an online community for engineering students, offering:

  • Free viva questions PDFs
  • Previous year question papers (PYQs)
  • Academic doubt solutions
  • Expert-guided solutions

Get the pro version for free by logging in!

Recent Badges

Awarded Badge User Avatar
Doubtly Earned a badge
2 hours ago Famous Question
Awarded Badge User Avatar
MrRam Earned a badge
4 days ago Popular Question
Awarded Badge User Avatar
Ritesh Maurya Earned a badge
4 days ago Notable Question

Most popular tags

jeemain geography-10th engineering physics semester 5 web computing

5.7k questions

5.1k answers

108 comments

648 users

Doubtly
...