How to find the total number of HTTP requests ?

@mit $ingh. July 09, 2015 Comments

We mentioned that we would need to find the total number of HTTP requests generated by a given URL – let’s figure out how to write some code that will do that for us. It’s clear that we must have some variable that maintains a total count of all HTTP requests, and this variable will be incremented as we come across more and more HTTP requests.

We know that images will be wrapped in an “img” tag – so if we just do a search for all img tags we can take a look at the src attribute, and find the size of any given image. For each image we find, we can increment the variable that holds the total count of the HTTP requests. We can also do the same for CSS files – they will be referenced inside “link” tags, and also for JavaScript files, which will be referenced inside “script” tags.

We will need to use the simple HTML DOM parser that we discussed earlier in order to find all of the references to CSS, Javascript, and image files. Here’s what the code looks like – note that we are using the simple HTML DOM library functionality to parse through the HTML. Also note that we are using a variable called $totalNumResources to hold the total number of resources, and another variable called $totalSize to hold the total size of all of the resources:

include('simple_html_dom.php');

$URL = $argv[1];

// Create DOM from URL or file
$html = file_get_html($URL);

// find all images!!
foreach($html->find('img') as $element){

	   $size = get_remote_file_size($element->src);
		
	   $totalSize = $totalSize + $size; 	
	   
	   $totalNumResources += 1;

	   /*
	   echo "Total Size So Far: $totalSize.\n"; 
	   
	   echo "total resources: $totalNumResources .\n"; 

           echo "IMAGE SIZE: $size.\n";

           echo "$element->src.\n";
       */
}

// find all CSS files
foreach($html->find('link') as $element)
{
    if (strpos($element->href,'.css') !== false) {

	   $size = retrieve_remote_file_size($element->href);
	   
	    echo "SIZE: $size.\n";

	    $totalSize = $totalSize + $size; 
	   	   
	    $totalNumResources += 1;
     }
}

// find all script tags
foreach($html->find('script') as $element)
{
  //make sure this is javascript
  if (strpos($element->src,'.js') !== false) {
    $size = get_remote_file_size($element->src);
	  
     echo " Javascript SIZE: $size.\n"; 

     $totalSize = $totalSize + $size; 		
	  	   
      $totalNumResources += 1;
   }
}

###
function get_remote_file_size($url) {

$headers = get_headers($url, 1);
    
if (isset($headers['Content-Length'])) 
       return $headers['Content-Length'];
    
    //checks for lower case "L" in Content-length:
if (isset($headers['Content-length'])) 
       return $headers['Content-length'];


}

 

  • http
  • http-request