During the web page loading process, numerous files are downloaded in the background, both statically and dynamically, based on different formats. Through the browser’s Network tab, you can access information such as which stage each file was downloaded, its size, and its content. Files can also be grouped according to their formats. However, to examine and/or analyze files offline outside of the browser, files are typically downloaded individually or evaluated via the Network panel. In this article, I will discuss an alternative method—using HAR—to capture requests.
During the page loading process, we can observe requests (GET, POST, etc.) and the responses sent or returned as a result of these requests, all via the Network tab.
When you right-click on a relevant request, either generally or according to a specific format, a Save all as HAR with content option appears. With this option, the relevant requests can be saved in HAR format (.har) to your computer. Below is an example HAR content containing JavaScript requests.
So, what is HAR?
HTTP Archive (HAR) Format
HTTP Archive (HAR) is a file format that results from logging a web browser’s interaction with a website in JSON format. This archive contains detailed performance data related to the requests (headers, metadata, cookies, form data, etc.) and can be exported externally1 2 3.
[mermaid] flowchart TD C[Log] C —>D[Creator] C —>E[Pages] C —>F[Entries] [/mermaid]
This format was developed by the Web Performance Working Group of the World Wide Web Consortium (W3C), but has never been published1.
The following is an example of a HAR archive content containing JavaScript requests.
{
"log": {
"version": "1.2",
"creator": {
"name": "WebInspector",
"version": "537.36"
},
"pages": [],
"entries": [
{
"_initiator": {
"type": "script",
"stack": {
"callFrames": [
{
"functionName": "send",
"scriptId": "65",
"url": "https://script.bugpilot.io/d1d055a0-ece1-4f91-9ac1-9ae73f43df0d/adopto.js",
"lineNumber": 572,
"columnNumber": 665
},
{
"functionName": "Ta",
"scriptId": "164",
"url": "https://www.clarity.ms/s/0.7.7/clarity.js",
"lineNumber": 1,
"columnNumber": 40543
},
{
"functionName": "",
"scriptId": "164",
"url": "https://www.clarity.ms/s/0.7.7/clarity.js",
"lineNumber": 1,
"columnNumber": 39956
},
{
"functionName": "",
"scriptId": "164",
"url": "https://www.clarity.ms/s/0.7.7/clarity.js",
"lineNumber": 1,
"columnNumber": 7773
},
{
"functionName": "",
"scriptId": "164",
"url": "https://www.clarity.ms/s/0.7.7/clarity.js",
"lineNumber": 1,
"columnNumber": 7878
},
{
"functionName": "o",
"scriptId": "164",
"url": "https://www.clarity.ms/s/0.7.7/clarity.js",
"lineNumber": 1,
"columnNumber": 6595
}
]
}
}
}]
// ...
}
}HAR files can also be used to download relevant files4 5, including all interactions that occur during the rendering of a website, and can be processed via various tools6 7 8.
Github. “swiss-army-scripts”. Under the har-js-analyzer/js-file-downloader.py script, it is possible to download files present in a HAR file (for example, this script handles JS-formatted files). Using har-js-analyzer/js-analyzer.py, a basic analysis of the relevant HAR content can be performed.
Basic HAR Analysis Process
The sample code snippet contains requests related to JS files, and the analysis process is also conducted within this context. Naturally, since the file format can be changed, the format option is removed, allowing the relevant operations to be performed for all requests.
…
import json
from haralyzer import HarParser, HarPage
with open('har_data.har', 'r') as f:
har_parser = HarParser(json.loads(f.read()))
print har_parser.browser
# {u'name': u'Firefox', u'version': u'25.0.1'}
print har_parser.hostname
# 'humanssuck.net'
for page in har_parser.pages:
assert isinstance(page, HarPage, None)
# returns True for eachUsing the Python snippet in the har-image-analyzer.py file, you can access information such as the number of entry entries, columns within each entry, summary statistics (min, max, mean, etc.) related to requests, image file names and sizes, group images, and compare total loading times.
- Number of entries: 38
- Number of image requests: 18
| File Name | Size |
|---|---|
| wikipedia.png | 13444 B |
| 100px-Salvador_Dal%C3%AD_1939.jpg | 4674 B |
| 150px-The_Simpsons_yellow_logo.svg.png | 6716 B |
| 100px-YuanEmperorAlbumKhubilaiPortrait.jpg | 3038 B |
| 68px-Ask_and_Embla_by_Robert_Engels.jpg | 4934 B |
| 250px-Sergei_Rachmaninoff_LOC_33969u.jpg | 18170 B |
| 48px-Notification-icon-Wikiquote.svg.png | 1322 B |
| 48px-Notification-icon-Wikisource-logo.svg.png | 1784 B |
| 48px-Notification-icon-Commons-logo.svg.png | 960 B |
| 48px-Notification-icon-Wiktionary-logo.svg.png | 4270 B |
| 48px-Notification-icon-Wikidata-logo.svg.png | 568 B |
| 48px-Notification-icon-Wikibooks-logo.svg.png | 2770 B |
| 48px-Notification-icon-Wikiversity-logo.svg.png | 1813 B |
| 48px-MediaWiki-2020-icon.svg.png | 2890 B |
| 48px-Notification-icon-Wikivoyage-logo.svg.png | 628 B |
| 48px-Notification-icon-Wikispecies-logo.svg.png | 1680 B |
| 48px-Notification-icon-Meta-logo.svg.png | 2010 B |
| 171px-Logo_original-t.png | 9212 B |
| File Name | Request Count | Total Size | Maximum Load Time | Minimum Load Time | Average Load Time |
|---|---|---|---|---|---|
| wikipedia.png | 1 | 13444 B | 318.45 ms | 318.45 ms | 318.45 ms |
| 100px-Salvador_Dal%C3%AD_1939.jpg | 1 | 4674 B | 522.38 ms | 522.38 ms | 522.38 ms |
| 150px-The_Simpsons_yellow_logo.svg | 1 | 4977 B | 285.14 ms | 285.14 ms | 285.14 ms |
| 220px-Leonhard_Euler_2.jpg | 1 | 13595 B | 308.75 ms | 308.75 ms | 308.75 ms |
| 1200px-Manhattan_wikivoyage_banner.jpg | 1 | 142069 B | 1.25 s | 1.25 s | 1.25 s |
| 2000px-BlankMap-World6.svg | 1 | 171246 B | 982.39 ms | 982.39 ms | 982.39 ms |
| 1400px-Rubik’s_cube.svg.png | 1 | 182729 B | 1.22 s | 1.22 s | 1.22 s |
| 2000px-Map_of_the_world_by_the_US_Gov_as_of_2016.svg | 1 | 710413 B | 1.54 s | 1.54 s | 1.54 s |
Footnotes
- HTTP Archive (HAR) format ↩ ↩2
- HAR (file format). Wikipedia ↩
- View request payload. Network features reference. Chrome Developer ↩
- HAR Viewer ↩
- Google Administrator Tool Kit. HAR Analyzer ↩
- Github. “swiss-army-scripts”. har-js-analyzer/js-file-downloader.py ↩
- Leonardo Faria. “How to Use HAR Files to Analyze Performance Over Time” ↩
- Rakesh Seal. “A Comprehensive Guide on HAR Files” ↩