Help identifying Anonymous Proxy

monkeymagix1 · January 23rd, 2011, 06:49 PM

I have created myself an AJAX / PHP based script that finds working proxies and tests them in real time (as less than 1% of most free proxy lists work at the time of use).

I am using some CURL code to make an HTTP request to a special page I have created that does a GEO code lookup on the IP to get the location of the IP as well as checking the various header values such as

HTTP_CLIENT_IP
REMOTE_ADDR
HTTP_X_FORWARDED_FOR
HTTP_X_FORWARDED
HTTP_FORWARDED_FOR
HTTP_FORWARDED
HTTP_VIA
...

and so on (about 20 or so different HTTP values for various proxies)

The page I am scanning does a check on these headers to see if it can spot that a proxy or gateway has been used. It checks whether the REMOTE_ADDR doesn't match the HTTP_VIA or one of the FORWARDED_FOR values I can spot a proxy has been used.

This is all well and good and when I run my scan on my proxy lists I get back a mixture of proxies that are Transparent (I can spot the proxy) or Anonymous (in that if one has been used I cannot spot it)

However in certain cases when I take one of these supposedly anonymous proxies and set my browser up to use the details for it and then request the same test page through Firefox I am getting a different result.

Instead of reporting it as anonymous as the initial report did when I used CURL it will show it as transparent (in that one of the HTTP headers has passed the originating IP details along).

Therefore I am trying to get my head around why this is happening.

Obviously I am using CURL PHP code for the initial test and a web Browser for the 2nd test so maybe there is something going on at the proxy server in detecting how its being accessed however I would like to know for certain what the issue is as I want to be able to take an IP:PORT and test it for its level of privacy if possible.

Could someone give me some tips, or pointers on where I am going wrong . If I am going about this totally the wrong way can someone point me in the right direction to how I can determine the level of privacy from an IP:PORT using PHP code (or C# or .NET or even PERL or any language!)

The CURL code I am using sets the useragent to the browser making the request and the code is below

Code:

$fp = curl_init();

if (version_compare($curlversion, '7.10.5', '>='))
{
	curl_setopt($fp, CURLOPT_ENCODING, '');
}
curl_setopt($fp, CURLOPT_URL, $url);
curl_setopt($fp, CURLOPT_HEADER, TRUE);
curl_setopt($fp, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($fp, CURLOPT_TIMEOUT, $timeout);
curl_setopt($fp, CURLOPT_CONNECTTIMEOUT, $timeout);
curl_setopt($fp, CURLOPT_SSL_VERIFYPEER , false );
curl_setopt($fp, CURLOPT_SSL_VERIFYHOST , false );

// set this to current useragent - or default to Firefox
curl_setopt($fp, CURLOPT_USERAGENT, $useragent);

if (!ini_get('open_basedir') && !ini_get('safe_mode') && version_compare($curlversion, '7.15.2', '>='))
{
	curl_setopt($fp, CURLOPT_FOLLOWLOCATION, 1);
	curl_setopt($fp, CURLOPT_MAXREDIRS, $maxredirs);
}

if($method=="POST"){
	curl_setopt($fp, CURLOPT_POST, TRUE); // POST
	curl_setopt($fp, CURLOPT_POSTFIELDS,$postdata); // POST DATA
}

// do we go through a proxy?
if(!empty($proxy)){
	list($ip,$port) = explode(":",$proxy,2);
			
	curl_setopt($fp, CURLOPT_PROXYPORT, $port); // proxy port
	curl_setopt($fp, CURLOPT_PROXY, $proxy); // proxy details in format IP:PORT
	curl_setopt($fp, CURLOPT_HTTPPROXYTUNNEL, TRUE); // tell curl to proxy tunnel
}

// return my html source into my array
$urlinfo['html'] = curl_exec($fp);

Any help would be much appreciated.