Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://mail.python.org/pipermail/python-dev/2009-February/086552.html below:

[Python-Dev] Googlebot and the mail.python.org python-dev?archive

• Previous message: [Python-Dev] Googlebot and the mail.python.org python-dev archive
• Next message: [Python-Dev] What type of object mmap.read_byte should return on py3k?
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On Sat, Feb 28, 2009 at 10:37:09AM +0000, Antoine Pitrou wrote:
> have the following HTML tag in them:
>     <META NAME="robots" CONTENT="noindex,follow">
> which explicitly instructs Web spiders *not* to index contents nor follow links.

I believe this makes spiders not index this page, but does follow
links.  Individual messages have "index,nofollow".

--amk

• Previous message: [Python-Dev] Googlebot and the mail.python.org python-dev archive
• Next message: [Python-Dev] What type of object mmap.read_byte should return on py3k?
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4