MEMEPh. ideas that are worth sharing...
 » rendc.org  » rendc.com  » notedc.com

 Blogs

Programming

Frameworks

In-depth understanding of browser caching mechanism

I. Introduction

Caching can be said to be a simple and efficient way to optimize performance. An excellent caching strategy can shorten the distance that web pages request resources, reduce latency, and because cached files can be reused, it can also reduce bandwidth and network load.

For a data request, it can be divided into three steps: initiating a network request, back-end processing, and browser response. Browser caching can help us optimize performance in the first and third steps. For example, if the cache is used directly without initiating a request, or if a request is initiated but the data stored in the backend is consistent with the frontend, then there is no need to send the data back, which reduces the response data.

In the following content, we will explore the browser caching mechanism through cache location, cache strategy, and practical application of cache strategy.

 

2. Cache location

There are four types of cache locations, and each has its own priority. When the cache is searched in turn and none of them hits, the network will be requested.

 

1. Service Worker

Service Worker is an independent thread running behind the browser and can generally be used to implement caching functions. When using Service Worker, the transport protocol must be HTTPS. Because request interception is involved in Service Worker, the HTTPS protocol must be used to ensure security. Service Worker's cache is different from other built-in caching mechanisms in browsers. It allows us to freely control which files are cached, how to match the cache, how to read the cache, and the cache is persistent.

Service Worker implements the caching function in three steps: First, you need to register the Service Worker, and then you can cache the required files after listening to the install event. Then the next time the user accesses, you can query whether there is a cache by intercepting the request. , if there is a cache, you can directly read the cache file, otherwise it will request the data.

When the service worker does not hit the cache, we need to call the fetch function to get the data. That is to say, if we do not hit the cache in the Service Worker, the data will be looked up according to the cache lookup priority. But whether we get the data from the Memory Cache or from a network request, the browser will show what we got from the Service Worker.

 

2. Memory Cache

Memory Cache is the cache in memory, which mainly contains the resources that have been captured in the current page, such as styles, scripts, pictures, etc. that have been downloaded on the page. Reading data in memory is definitely faster than disk. Although the memory cache is efficient in reading, the cache persistence is very short and will be released as the process is released. Once we close the tab page, the in-memory cache is freed.

So since the memory cache is so efficient, can we keep all data in memory?
it's out of the question. The memory in the computer must be much smaller than the hard disk capacity. The operating system needs to carefully calculate the use of memory, so there must be not much memory that can be used by us.

When we visit the page and refresh the page again, we can find that a lot of data comes from the memory cache

An important cached resource in the memory cache is <link rel="prefetch">the resource downloaded by preloader-related instructions (for example). It is well known that the related instructions of the preloader are already one of the common methods of page optimization. It can parse the js/css file while requesting the next resource from the network.

It should be noted that the memory cache does not care what the value of the HTTP cache header Cache-Control of the returned resource is when caching resources. At the same time, the matching of resources is not only for URL matching, but also for Content-Type, CORS. Check other features.

 

3. Disk Cache

Disk Cache is the cache stored in the hard disk. The reading speed is slower, but everything can be stored on the disk. Compared with the Memory Cache, it is better than the Memory Cache in terms of capacity and storage timeliness.

Among all browser caches, the coverage of Disk Cache is basically the largest. It will judge which resources need to be cached according to the fields in the HTTP Herder, which resources can be used directly without request, and which resources have expired and need to be re-requested. And even in the case of cross-site, once the resource of the same address is cached by the hard disk, it will not request data again. Most of the cache comes from Disk Cache. We will introduce the cache field in the HTTP protocol header in detail below.

What files does the browser drop into memory? Which ones go to the hard drive?
There are different opinions on the Internet about this, but the following opinions are more reliable:

 

4. Push Cache

Push Cache (push cache) is the content in HTTP/2, it will be used when none of the above three caches are hit. It only exists in the session (Session) and is released once the session ends, and the cache time is also very short, only about 5 minutes in the Chrome browser, and it does not strictly implement the cache instructions in the HTTP header.

Push Cache has very little information that can be found in China, also because HTTP/2 is not popular enough in China. The recommended reading here Jake Archibaldis HTTP/2 push is tougher than I thought . Several conclusions in the article:

If none of the above four caches are hit, then only requests can be made to obtain resources.

Then for performance considerations, most interfaces should choose a good caching strategy. Usually, browser caching strategies are divided into two types: strong caching and negotiated caching, and caching strategies are implemented by setting HTTP headers.

 

Third, the cache process analysis

The communication mode between the browser and the server is the response mode, that is: the browser initiates an HTTP request - the server responds to the request, so how does the browser determine whether a resource should be cached, and how to cache it? After the browser initiates the request to the server for the first time and gets the request result, it stores the request result and the cache ID in the browser cache. The browser's processing of the cache is determined according to the response header returned when the resource is requested for the first time.

The above two conclusions are the key to the browser caching mechanism. It ensures the cache storage and reading of each request. If we understand the usage rules of the browser cache, all the problems will be solved. This article will also focus on This point is analyzed in detail. In order to facilitate everyone's understanding, here we divide the caching process into two parts according to whether it is necessary to re-initiate an HTTP request to the server, namely strong caching and negotiated caching.

 

Fourth, strong cache

Strong cache: No request is sent to the server, and resources are read directly from the cache. In the Network option of the chrome console, you can see that the request returns a status code of 200, and the Size displays from disk cache or from memory cache. Strong caching can be achieved by setting two HTTP headers: Expires and Cache-Control.

 

1. Expires

The cache expiration time is used to specify the time when the resource expires, which is a specific time point on the server side. That is to say, Expires=max-age + request time, which needs to be used in conjunction with Last-modified. Expires is a web server response message header field, which tells the browser that the browser can directly fetch data from the browser cache before the expiration time without having to request again.

Expires is a product of HTTP/1 and is limited by local time. If the local time is modified, it may cause cache invalidation . Expires: Wed, 22 Oct 2018 08:41:00 GMTIndicates that the resource will expire after Wed, 22 Oct 2018 08:41:00 GMT and needs to be requested again.

 

2. Cache-Control

In HTTP/1.1, Cache-Control is the most important rule, mainly used to control web page caching. For example, Cache-Control:max-age=300at that time, it means that the resource will be loaded again within 5 minutes of the correct return time of the request (the browser will also record it), and the strong cache will be hit.

Cache-Control can be set in request headers or response headers, and can be combined with various directives:

public: All content will be cached (both client and proxy servers cacheable) . Specifically, the response can be cached by any intermediate node, such as Browser <-- proxy1 <-- proxy2 <-- Server, the intermediate proxy can cache resources, for example, the next time the same resource is requested, proxy1 directly sends the cached content to the Browser instead of Don't ask proxy2 anymore.

private: Only the client can cache all content , the default value of Cache-Control. Specifically, it means that the intermediate node does not allow caching. For Browser <-- proxy1 <-- proxy2 <-- Server, the proxy will honestly send the data returned by the Server to proxy1 and will not cache any data by itself. When the Browser requests again next time, the proxy will do a good job of forwarding the request instead of sending the cached data to itself.

no-cache: The client caches the content. Whether to use the cache needs to be verified by negotiating the cache. Indicates that the cache control method of Cache-Control is not used for pre-validation, but the Etag or Last-Modified field is used to control the cache. Note that the name no-cache is a bit misleading. After setting no-cache, it does not mean that the browser no longer caches data, but when the browser uses cached data, it needs to confirm whether the data is still consistent with the server.

no-store: All content will not be cached, i.e. no forced cache nor negotiated cache is used

max-age: max-age=xxx (xxx is numeric) indicates that the cached content will be invalidated after xxx seconds

s-maxage (unit is s): Same as max-age, it only takes effect in the proxy server (such as CDN cache). For example, when s-maxage=60, in these 60 seconds, even if the content of the CDN is updated, the browser will not make a request. max-age is for normal caching, while s-maxage is for proxy caching. s-maxage takes precedence over max-age. If s-maxage exists, the max-age and Expires headers are overwritten.

max-stale: The maximum expiration time that can be tolerated. The max-stale directive indicates that the client is willing to receive a response that has expired. If the value of max-stale is specified, the maximum tolerated time is the corresponding number of seconds. If not specified, it indicates that the browser is willing to receive a response with any age (age represents the difference between the time when the response was generated or confirmed by the origin site and the current time).

min-fresh: The minimum freshness that can be tolerated. min-fresh indicates that the client is unwilling to accept a response whose freshness is not greater than the sum of the current age plus the time set by min-fresh.

We can see from the figure that we can use multiple instructions together to achieve multiple purposes. For example, we hope that resources can be cached, and both the client and the proxy server can be cached, and the cache invalidation time can be set, and so on.

3. Comparison of Expires and Cache-Control

In fact, there is not much difference between the two. The difference is that Expires is a product of http1.0, and Cache-Control is a product of http1.1. If the two exist at the same time, Cache-Control has a higher priority than Expires; in some cases, HTTP1 is not supported. .1 environment, Expires will come into play. So Expires is actually an outdated product, and its existence at this stage is just a way of writing compatibility.
The strong cache determines whether the cache is based on whether it exceeds a certain time or a certain period of time, and does not care whether the server-side file has been updated, which may cause the loaded file to not be the latest content on the server-side, so how do we know the server-side content? Has an update happened yet? At this point we need to use a negotiated caching strategy.

 

5. Negotiate cache

Negotiating caching is a process in which the browser initiates a request to the server with the cache ID after forcing the cache to fail, and the server decides whether to use the cache based on the cache ID. There are two main cases:


Negotiated caching can be achieved by setting two HTTP headers: Last-Modified and ETag.

1. Last-Modified & If-Modified-Since

When the browser accesses the resource for the first time, when the server returns the resource, it adds the Last-Modified header to the response header. The value is the last modification time of the resource on the server. The browser caches the file and header after receiving it.

Last-Modified: Fri, 22 Jul 2016 01:47:00 GMT

The next time the browser requests this resource, the browser detects that there is a Last-Modified header, so it adds the If-Modified-Since header, and the value is the value in Last-Modified; the server receives this resource request again, according to If- The value in Modified-Since is compared with the last modification time of the resource in the server. If there is no change, return 304 and an empty response body and read it directly from the cache. If the time of If-Modified-Since is less than the last modification time of the resource in the server Modification time, indicating that the file has been updated, so return the new resource file and 200

But Last-Modified has some drawbacks:

Since whether the cache is still insufficient is determined based on the file modification time, can the cache strategy be determined directly based on whether the file content is modified? So, in HTTP/1.1 came ETagandIf-None-Match

 

2. ETag and If-None-Match

Etag is a unique identifier (generated by the server) of the current resource file that is returned when the server responds to the request. As long as the resource changes, the Etag will be regenerated. The next time the browser loads resources and sends a request to the server, it will put the Etag value returned last time into the If-None-Match in the request header. The server only needs to compare the If-None-Match sent by the client with its own server. If the ETag of the resource is consistent, it can be well judged whether the resource has been modified relative to the client. If the server finds that the ETag does not match, it will directly send the new resource (including the new ETag) to the client in the form of a regular GET 200 return packet; if the ETag is consistent, it will directly return 304 to notify the client directly Just use local cache.

 

3. Comparison between the two:

The time unit of Last-Modified is seconds. If a file is changed multiple times within 1 second, then their Last-Modified does not actually reflect the modification, but the Etag will change every time to ensure accuracy; if it is load balanced The Last-Modified generated by each server may also be inconsistent.

 

6. Cache mechanism

Mandatory caching takes precedence over negotiated caching. If mandatory caching (Expires and Cache-Control) takes effect, the cache is used directly. If it does not take effect, negotiated caching is performed (Last-Modified / If-Modified-Since and Etag / If-None-Match) , the negotiated cache is determined by the server whether to use the cache. If the negotiated cache is invalid, then the cache representing the request is invalid, return 200, return the resource and cache ID, and then store it in the browser cache; if it is valid, return 304 and continue to use the cache.

Seeing this, I don't know if you have such a question: If no cache policy is set, what will the browser do?

In this case, the browser will use a heuristic algorithm, usually taking the Date in the response header minus 10% of the Last-Modified value as the cache time.

 

Seven, the actual scene application cache strategy

1. Frequently changing resources

Cache-Control: no-cache

For frequently changing resources, you need to use Cache-Control: no-cacheto make the browser request the server every time, and then cooperate with ETag or Last-Modified to verify whether the resource is valid. Although this approach does not save the number of requests, it can significantly reduce the size of the response data.

2. Infrequently changing resources

Cache-Control: max-age=31536000

Usually when dealing with such resources, configure their Cache-Control with a large value max-age=31536000(one year), so that subsequent requests by the browser for the same URL will hit the forced cache. In order to solve the problem of updating, it is necessary to add dynamic characters such as hash, version number, etc. to the file name (or path), and then change the dynamic characters, so as to achieve the purpose of changing the reference URL and invalidate the previous mandatory cache (in fact, it is not immediately Ineffective, just no longer used).
Class libraries available online (eg jquery-3.3.1.min.js, lodash.min.jsetc.) use this pattern.

 

8. The impact of user behavior on browser cache

The so-called influence of user behavior on browser cache refers to what kind of cache policy is triggered when the user operates the browser. There are mainly 3 kinds:

 

Deep dive into browser storage What happens from URL input to page presentation