H! I'm busy with testing python and now i'm trying to check if a url makes a forward to a other location with the same content. So it will be possible to scan unique website's. I already made these checks: the html forward: <meta http-equiv="refresh" content="0;URL=othersite.com"> the header: Content-Location: othersite.com the url - check: if '-': check if 'sitename.com' has not the same content as site-name.com the problem: There are still website's forwarding to a other location with the same content but they overrule the 3 checks above. Thanks very Much, GC-Martijn
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4