Definition of lastmodified for html bitrix pages. The Last-Modified header speeds up the indexing of new pages significantly. How caching works in browsers
In area search engine optimization There are a lot of different myths circulating around websites (SEO). Some of them have a basis, some of them came from nowhere. In this note we will look at one of them - using the last-Modified response header.
Some time ago we received a document entitled “Ingate Recommendations for Web Studios on Promoted Sites.” And one of the “recommendations” was the following:
After a redesign or on a new site being developed, the date of the last modification of the site pages (Last Modified) must be indicated.
To add information about the date of the last modification of pages to a site in PHP, you need to go to the very beginning source code insert a script into each page
header("Last-Modified: " . date("D, d M Y H:i:s", time()) . " GMT");
?>
It was this wild nonsense, this utter nonsense and frankly crazy code that prompted me to write this note. Here I will try to explain what Last-Modified is, why it is needed and how browsers and search engines use it.
What is Last-Modified
When transmitting information to the client (browsers or search robot), the web server reports quite a lot of additional data. They can be viewed in the browser console, for example:
configure the server to issue correct response headers (for example, if the page does not exist, issue a 404 error, and if an If-Modified-Since request is received, then issue a 304 code if the page has not been changed since the date specified in the request).
You can also see that if the server does not respond in any way to a conditional GET request, then it is no different from a regular request. That is, the Last-Modified header with current time, besides, incorrectly formed ones (hello Integgate!) are not needed at all!
So is Last-Modified necessary or not?
Generally necessary. But it is important to understand that it is not the header itself that plays any role, but the entire conditional request scenario, which must be fully implemented by the site. It is in this case that we will get a high speed of site indexing.
But it is often very difficult to implement this in a ready-made CMS. This may require quite significant changes to the code of the CMS itself.
Although for a number of CMS this can be achieved by enabling page caching. If the CMS caches pages, creating and serving essentially static files, then the web server itself will respond correctly to conditional requests. For example, in WordPress this can be achieved using the WP Super Cache plugin:
Let's check it in action. I enabled this plugin, opened the browser in anonymous mode and made two requests for the same page. It is clearly seen that the second answer is correct - 304 Not Modified:
Instead of a conclusion
Thus, we have dealt with the Last-Modified header. First, it must convey information about the date and time the document was actually modified. Secondly, the server’s response to a conditional request with the If-Modified-Since header is extremely important.
Well, listen less to SEOs who don’t know the basics of how the Internet works.
Syntax
Last-Modified:Directives
GMT
Greenwich Mean Time. HTTP dates are always expressed in GMT, never in local time.
ExamplesLast-Modified: Wed, 21 Oct 2015 07:28:00 GMT
| Specifications | Specification |
|---|---|
| Title | RFC 7232, section 2.2: Last-Modified Hypertext Transfer Protocol |
(HTTP/1.1): Conditional Requests
Browser compatibility
The compatibility table in this page is generated from structured data. If you"d like to contribute to the data, please check out https://github.com/mdn/browser-compat-data and send us a pull request.
| Update compatibility data on GitHub | Desktop | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mobile | Chrome | Edge | Firefox | Internet Explorer | Opera | Safari | Android webview | Chrome for Android | Firefox for Android | Opera for Android | Safari on iOS | |
| Samsung Internet | Last-Modified | Chrome Full support Yes | Edge Full support 12 | Firefox Full support Yes | IE Full support Yes | Opera Full support Yes | Safari Full support Yes | WebView Android Full support Yes | Chrome Android Full support Yes | Firefox Android Full support Yes | Opera Android Full support Yes | Safari iOS Full support Yes |
Samsung Internet Android Full support Yes The Last-Modified HTTP header tells the client the time last change page (object). If the client (browser,) received the Last-Modified header, then the next time the address is accessed, provided that the page (object) is in the local cache, it will add an If-Modified-Since question (whether the page has changed since the date received in Last-Modified). In turn, the server, having received the If-Modified-Since request, must check the received timestamp with the time the page was last modified and, if the page has not changed, respond with 304 Not Modified.
Traffic Saving
If the page has not changed, the server will stop transmitting data after sending headers with the 304 Not Modified code; the page body, images and other objects will not be transmitted.
Reducing server load
Correct implementation of checking the last page modification time can significantly (up to 30% or more) reduce the load on the server. Correct implementation means checking the time before the page is generated on a dynamic site. In this case, all actions to generate the page (querying content from the database, parsing templates, receiving comments, etc.) will not be performed. This is especially true for sites with high traffic and a long user visit. Example: a user is on a sports news site and is constantly updating home page awaiting publication of the match result. In a few minutes, a page can be requested and received dozens of times. If the Last-Modified header is sent and the If-Modified-Since request is processed correctly, then the page will actually be submitted once, and all subsequent requests will receive a 304 Not Modified response.
Speed up indexing by search engines
Search engines, through their webmaster guidelines, recommend sending the Last-Modified header and properly handling If-Modified-Since .
Make sure your web server supports the If-Modified-Since HTTP header. This header will allow the web server to tell Google whether the site's content has changed since the last time it was crawled. Support for this feature will reduce the load on throughput and costs.
Make sure your http headers are correct. In particular, what is important is the content of the response that the server gives to the if-modified-since request. The Last-Modified header must indicate the correct date the document was last modified. If the server does not display the last modification date of the document, your site will be indexed, but:
- the search results will not show the date next to the pages of your site;
- when sorting by date, the site will not be visible to most users;
- the robot will not be able to obtain information about whether the site page has been updated since the last indexing. And since the number of pages a robot receives from a site in one visit is limited, changed pages will be reindexed less often.
Syntax
If-Modified-Since:Directives
GMT
Greenwich Mean Time. HTTP dates are always expressed in GMT, never in local time.
If-Modified-Since: Wed, 21 Oct 2015 07:28:00 GMTLast-Modified: Wed, 21 Oct 2015 07:28:00 GMT
| Specifications | Specification |
|---|---|
| RFC 7232, section 3.3: If-Modified-Since | Hypertext Transfer Protocol (HTTP/1.1): Conditional Requests |
(HTTP/1.1): Conditional Requests
Browser compatibility
The compatibility table in this page is generated from structured data. If you"d like to contribute to the data, please check out https://github.com/mdn/browser-compat-data and send us a pull request.
| Update compatibility data on GitHub | Desktop | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mobile | Chrome | Edge | Firefox | Internet Explorer | Opera | Safari | Android webview | Chrome for Android | Firefox for Android | Opera for Android | Safari on iOS | |
| If-Modified-Since | Last-Modified | Chrome Full support Yes | Edge Full support 12 | Firefox Full support Yes | IE Full support Yes | Opera Full support Yes | Safari Full support Yes | WebView Android Full support Yes | Chrome Android Full support Yes | Firefox Android Full support Yes | Opera Android Full support Yes | Safari iOS Full support Yes |
I didn’t have any sadness until I installed it today SEO plugin Scan. A good plugin that shows what Google may not like about your site and server. And there was one thing I didn’t like about my site for sure - the lack of last-modified .
A little theory, I just found out about it today. Last-modified is an http header that sends a signal about whether there have been changes on such and such a page or not, sending an If-Modified-Since response with a 304 code. Something like this...
Although without all this the site will be indexed, as Yandex says, there are also significant disadvantages in the absence of this header:
- the search results will not show the date next to the site pages;
- when sorting by date, the site will not be visible to users;
- the robot will not be able to obtain information about whether the site page has been updated since the last indexing. And since the number of pages the robot receives from the site in one visit is limited, changed pages will be re-indexed less often.
As you can see, there are many disadvantages, and first of all, it is not very fast indexing and re-indexing of the site. That is, you made a USEFUL change to the article, and now I make a lot of them every day, and Yandex and Google can see It's only in a couple of months! No, we don’t need this kind of hockey, now I’ll smash myself into pieces and fix this shortcoming!
How to check last-modified?
You can check different ways, in wordpress the easiest thing is to install the above plugin SEO Scan, it gave me this:
It’s also acting up, but that’s another story, then I’ll dig deeper into that too. You can also check last-modified through a service of the same name last-modified.com
As you can see, it also shows the absence of a response If-Modified-Since - Last Modified not found!
Or you can use the Yandex service Checking the server response
It's the same here - HTTP status code:“304 Not Modified” means the system is not working. Although after some time the obvious error disappeared, the 304 response code never appeared.
I immediately became interested, does this even work for anyone? I decided to check this with those bloggers who themselves wrote manuals on this topic:
Yes, the guy from the TOP found Last-Modified for this request, which means it’s possible for me too. By the way, his article lists many ways to configure Last Modified, let's try them, although I found others.
How to configure last-modified?
I will describe all the methods that I tried and immediately say whether it worked for me or not. But keep in mind that if it doesn’t work for me, it doesn’t mean it won’t work for you.
1. The first method is to install the plugin
The easiest way for those who have WordPress is to install plugins designed for this, for example: Add Headers, If Modified Since or If Modified Since Header.
None of these plugins worked for me, I don’t know why. Maybe it will work for you?
2. Second method - PHP code
Service last-modified.com, described above, suggests placing the following php code in the header.php file, at the very beginning:
= $LastModified_unix) ( header($_SERVER["SERVER_PROTOCOL"] . " 304 Not Modified"); exit; ) header("Last-Modified: ". $LastModified);
?>
It didn't help me either. I also suggest trying to place it in the index.php file in the root of the site, but this didn’t help me either.
3. The third method is writing in .htaccess If you have an Apache server, then most likely this will work for you. Needed in file
.htaccess in the root of the site paste the code:
RewriteRule .* - RewriteRule .* -
It didn’t work for me, because I have a combination of Ngnix + Apache and all these manipulations with .htaccess most often do not work.
After all this torment I thought: why am I the only one suffering? I’ll write to the hosting support service, the people there are competent, maybe they can give me some advice? I wrote and am sitting and waiting.
I received a response from technical support:
It's simple. The pages of your site are generated by PHP scripts that do not display this header to the browser.
The web server (nginx) for static files (jpg, gif, css, js, etc.) displays the date of the last modification of the file in the last-modified header. Examples in screenshots.