R1:76ed0c7ff7be

Disallow webcrawlers to follow Paste line number anchor links

Summary:
Paste provides line anchor links in every single line of a paste.
If webcrawlers follow these links, they index the very same Paste again.
Thus disallow in robots.txt to reduce unneeded traffic and indexing time.

Closes T15662

Test Plan:
Go to `/robots.txt` in the web browser.
Cross fingers that more webcrawlers abide by RFC 9309.

Reviewers: O1 Blessed Committers, valerio.bozzolan

Reviewed By: O1 Blessed Committers, valerio.