Learn more about robots.txt
We made a video about how Google handles the robots.txt file. You can watch it if you want:
This answers a couple questions such as:
- Why is my url showing up in Google when I blocked it in robots.txt? Did you fetch that url?
- How do I make that url disappear from Google?
I hope the video helps if you have questions.
Original post: http://www.mattcutts.com/blog/robots-txt-remove-url/
