I have created myself an AJAX / PHP based script that finds working proxies and tests them in real time (as less than 1% of most free proxy lists work at the time of use).
I am using some CURL code to make an HTTP request to a special page I have created that does a GEO code lookup on the IP to get the location of the IP as well as checking the various header values such as
and so on (about 20 or so different HTTP values for various proxies)
The page I am scanning does a check on these headers to see if it can spot that a proxy or gateway has been used. It checks whether the REMOTE_ADDR doesn't match the HTTP_VIA or one of the FORWARDED_FOR values I can spot a proxy has been used.
This is all well and good and when I run my scan on my proxy lists I get back a mixture of proxies that are Transparent (I can spot the proxy) or Anonymous (in that if one has been used I cannot spot it)
However in certain cases when I take one of these supposedly anonymous proxies and set my browser up to use the details for it and then request the same test page through Firefox I am getting a different result.
Instead of reporting it as anonymous as the initial report did when I used CURL it will show it as transparent (in that one of the HTTP headers has passed the originating IP details along).
Therefore I am trying to get my head around why this is happening.
Obviously I am using CURL PHP code for the initial test and a web Browser for the 2nd test so maybe there is something going on at the proxy server in detecting how its being accessed however I would like to know for certain what the issue is as I want to be able to take an IP:PORT and test it for its level of privacy if possible.
Could someone give me some tips, or pointers on where I am going wrong . If I am going about this totally the wrong way can someone point me in the right direction to how I can determine the level of privacy from an IP:PORT using PHP code (or C# or .NET or even PERL or any language!)
The CURL code I am using sets the useragent to the browser making the request and the code is below
$fp = curl_init();
if (version_compare($curlversion, '7.10.5', '>='))
curl_setopt($fp, CURLOPT_ENCODING, '');
curl_setopt($fp, CURLOPT_URL, $url);
curl_setopt($fp, CURLOPT_HEADER, TRUE);
curl_setopt($fp, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($fp, CURLOPT_TIMEOUT, $timeout);
curl_setopt($fp, CURLOPT_CONNECTTIMEOUT, $timeout);
curl_setopt($fp, CURLOPT_SSL_VERIFYPEER , false );
curl_setopt($fp, CURLOPT_SSL_VERIFYHOST , false );
// set this to current useragent - or default to Firefox
curl_setopt($fp, CURLOPT_USERAGENT, $useragent);
if (!ini_get('open_basedir') && !ini_get('safe_mode') && version_compare($curlversion, '7.15.2', '>='))
curl_setopt($fp, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($fp, CURLOPT_MAXREDIRS, $maxredirs);
curl_setopt($fp, CURLOPT_POST, TRUE); // POST
curl_setopt($fp, CURLOPT_POSTFIELDS,$postdata); // POST DATA
// do we go through a proxy?
list($ip,$port) = explode(":",$proxy,2);
curl_setopt($fp, CURLOPT_PROXYPORT, $port); // proxy port
curl_setopt($fp, CURLOPT_PROXY, $proxy); // proxy details in format IP:PORT
curl_setopt($fp, CURLOPT_HTTPPROXYTUNNEL, TRUE); // tell curl to proxy tunnel
// return my html source into my array
$urlinfo['html'] = curl_exec($fp);
Any help would be much appreciated.