1 answer
Best answer
robots.txt is a text file that instructs web robots (search engine robots) about how to crawl pages on their website. It is used to manage crawler traffic to the website.
Anonymous
· 10 Aug 2022
Report
Was this answer helpful?
Thanks for the feedback.
Sign up free to save answers and ask your own doubts
Your answer
Log in or create a free account to answer this question.