Calculating Website Age & Tracking Changes With URL Dater

NIXINTEL: “How do you know how old a webpage really is? A core part of website investigation is determining precisely when site was created or when an article was published. For most sites this is rarely an issue of concern, but for websites that are intentionally misleading it can be more challenging. Websites created for fraud, phishing, disinformation or other deceptive purposes frequently misrepresent the age of the site in order to appear more established and authoritative than they really are. It’s also helpful to be able to find out exactly when an article was published and detect subsequent modifications. There are several ways to do this. Article publication dates might be obvious, but it is also trivial to edit publication timestamps and they cannot be relied upon. The Internet Archive can also be a useful indicator, but it has gaps in coverage and is slow to query. In any case a first crawl of a website will by definition always lag the actual creation of the site so it’s better to go upstream to find more reliable signals.

• To make this process quicker and easier I’ve put together URL Dater from a collection of old Python scripts that I’ve used for a few years but with a web interface to make the tooling and results a little easier to work with. URL Dater collects three sources of information to validate the age of a web page. First it uses RDAP (the replacement for WHOIS) to find out when the original domain was first registered and also when the registration was last updated. Secondly it uses certificate transparency logs to determine when the very first SSL certificate for that website was issued. Finally it connects to the target URL and reads the Last-Modified header timestamp from any images that are embedded on the page. These data points are combined together to produce a timeline showing when the site and its content were created or modified…”