Simple Html Dom库本地正常,多数付费线上服务器无法使用求助
我用Curl做网页抓取,搭配Simple Html Dom库解析HTML,这个库用起来挺简洁顺手的。但现在碰到个头疼的问题:本地服务器跑起来完全正常,000webhost主机也能正常工作,但在GoDaddy、Namecheap这些付费主机,还有Hostinger的付费套餐上都没法正常运行。联系GoDaddy客服,他们说这不是主机的问题,让我找开发者。我查了好多资料,试着启用了PHP的mbstring扩展,但问题还是没解决,搜索数据的时候总是返回空数组。试了各种方法都没搞定,也不知道问题到底出在哪。
我给你梳理几个大概率的方向,你可以逐一排查:
Curl请求被拦截或主机限制
很多付费主机可能会限制Curl的某些参数,或者目标站点https://allcstech.com/tele/search-result.php检测到非常规请求直接拦截了。你可以给Curl加更多模拟浏览器的参数,比如:curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); // 跟随页面跳转 curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // 部分主机SSL配置有问题,先关闭验证试试 curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false); curl_setopt($ch, CURLOPT_REFERER, "https://allcstech.com/tele/"); // 设置来源页,模拟正常访问流程另外,你可以把
CURLOPT_TIMEOUT从40调大到60,有些付费主机网络延迟高,超时时间不够会导致请求失败。Simple HTML Dom的编码不匹配
目标站点的HTML编码可能和你的PHP环境编码不一致,导致解析失败。你可以在加载HTML前先统一转码:$html = mb_convert_encoding($html, 'UTF-8', 'auto'); $dom->load($html);重点提醒:你先别着急解析,先把
$html打印出来看看——如果$html是空的,那问题出在Curl请求环节,和解析没关系。主机的PHP环境权限/限制
有些付费主机禁用了allow_url_fopen,或者你用的cookie.txt没有写入权限。你可以把cookie.txt权限改成777试试,或者干脆不用文件存储cookie,改用内存cookie:curl_setopt($ch, CURLOPT_COOKIEJAR, ""); // 启用内存cookie curl_setopt($ch, CURLOPT_COOKIEFILE, "");还有,你别再用
error_reporting(0)了,开启错误报告才能看到具体问题:error_reporting(E_ALL); ini_set('display_errors', 1);这样能直接看到是Curl报错还是Simple HTML Dom报错,定位问题快很多。
目标站点的反爬策略
说不定目标站点对不同IP/地区的请求返回不同页面?比如给线上主机返回反爬页面,给本地返回正常内容。你可以把线上主机获取到的$html打印出来,看看是不是和本地的页面结构一致,比如td的索引是不是还是3、5、7这些。
我把上面的优化点整合到你的代码里了,你可以直接测试:
<?php // 开启错误报告,方便排查问题 error_reporting(E_ALL); ini_set('display_errors', 1); $msg = ""; $firstname = $lastname = $cnic = $number = $mod = $address = $address1 = $city = $fullname = $otherdata = $otherdata1 = ""; if (isset($_POST['submit'])) { $search = $_POST['cnic']; if (strlen($search) < 12) { $search = substr($search, 1); } $url = "https://allcstech.com/tele/search-result.php"; $ch = curl_init(); curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']); curl_setopt($ch, CURLOPT_TIMEOUT, 60); // 延长超时时间 // 优化Curl配置,模拟浏览器正常请求 curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false); curl_setopt($ch, CURLOPT_REFERER, "https://allcstech.com/tele/"); // 改用内存cookie,避免文件权限问题 curl_setopt($ch, CURLOPT_COOKIEJAR, ""); curl_setopt($ch, CURLOPT_COOKIEFILE, ""); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_POSTFIELDS, "cnnum=$search"); $html = curl_exec($ch); // 检查Curl是否执行成功 if(curl_errno($ch)){ $msg = "Curl Error: ".curl_error($ch); } else { include 'dfapi/simple_html_dom.php'; $dom = new simple_html_dom(); // 统一编码后加载HTML $html = mb_convert_encoding($html, 'UTF-8', 'auto'); $dom->load($html); $firstname = $dom->find('td',3); $lastname = $dom->find('td',5); $cnic = $dom->find('td',7); $number = $dom->find('td',1); $mod = "<strong>0</strong>".$number; $address = $dom->find('td',9); // 修正判断逻辑,所有字段都为空才提示无记录 if(empty($firstname) && empty($lastname) && empty($cnic) && empty($number)) { $msg = "No record Found"; } foreach($dom->find('table') as $e) $otherdata = $e; $otherdata1 = $dom->find('table',1); } curl_close($ch); // 记得关闭Curl资源,避免内存泄漏 } ?> <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>DataFinder</title> <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css" integrity="sha384-ggOyR0iXCbMQv3Xipma34MD+dH/1fQ784/j6cY/iJTQUOhcWr7x9JvoRxT2MZw1T" crossorigin="anonymous"> <meta name="viewport" content="width=device-width, initial-scale=1"> <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css"> <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script> </head> <style> @font-face { font-family: Jameel Noori Nastaleeq; src: url('Jameel Noori Nastaleeq.ttf'); } img[src*="https://cdn.000webhost.com/000webhost/logo/footer-powered-by-000webhost-white2.png"] { display: none;} body { font-family: Arial; } * { box-sizing: border-box; } table { color: white; text-transform: capitalize; } td { text-align: right; margin-left: 20px; } form.example input[type=number] { padding: 10px; font-size: 17px; border: 1px solid grey; float: left; width: 80%; background: #f1f1f1; } form.example button { float: left; width: 20%; padding: 10px; background: #2196F3; color: white; font-size: 17px; border: 1px solid grey; border-left: none; cursor: pointer; } form.example button:hover { background: #0b7dda; } form.example::after { content: ""; clear: both; display: table; } /* Hide HTML5 Up and Down arrows. */ input[type="number"]::-webkit-outer-spin-button, input[type="number"]::-webkit-inner-spin-button { -webkit-appearance: none; margin: 0; } input[type="number"] { -moz-appearance: textfield; } #alert { font-family: Jameel Noori Nastaleeq, sans-serif;color: #fff } </style> <script> $(document).ready(function() { $("#hide").hide(); $("#again").hide(); $("#show").click(function() { $("#hide").show(); $("#show").hide(); $("#again").show(); }); $("#again").click(function() { $("#hide").hide(); $("#again").hide(); $("#show").show(); }); }); </script> <body style="background-color: orange"> <div class="container mt-2"> <span id="alert"><span id="again" style="cursor: pointer;"> (Hide this) </span> <span id="show" style="cursor: pointer;">...</span>ڈیٹافائنڈر گورنمنٹ کی طرف سے بلیک لسٹ میں ڈالا <span id="hide">گیا،جس کی وجہ سے ہمیں سرور میں وی پی این لگانا پڑا،وی پی این کی وجہ سے ڈیٹا سرچ میں ٹائم لگ سکتا ہے۔کسی بھی نمبر یا آئی ڈی کارڈ کا ڈیٹآ دو یا تین مرتبہ سرچ کرنے سے شو نہ ہو تو مطلب اس کا ڈیٹا نہیں ہے۔ </span></span> <form action="numdb.php" method="post" class="example"> <input type="number" placeholder="03xxxxxxxxx" name="cnic" autocomplete="off" minlength="10" value="<?php if(isset($_POST['submit'])) echo $_POST['cnic']; ?>"> <button type="submit" name="submit"><i class="fa fa-search"></i></button> </form> <?php echo $msg; ?> <div class="mt-4"> <strong class="float-left text-white">Name:</strong> <span class="float-right text-white"><?php echo $firstname." ".$lastname." ".$fullname; ?></span> <br><br> <strong class="float-left text-white">CNIC:</strong> <span class="float-right text-white"><?php echo $cnic; ?></span> <br><br> <strong class="float-left text-white">Number:</strong> <span class="float-right text-white"><?php echo $mod; ?></span> <br><br> <strong class="float-left text-white">Address:</strong> <span class="float-right text-white"><?php echo $address." ".$address1." ".$city; ?></span> <br><br><br> <strong><?php echo $otherdata; ?></strong> <br><br> <hr> <strong><?php echo $otherdata1; ?></strong> </div> </div> <center> <div style="color: white;size: 15px;margin: 10px"> <span> Install DataFinder Application and Get a lot other services like: </span> <ul> <li>Family Tree</li> <li>Cnic Copy</li> <li>Live Location</li> <li>Call History</li> <li>IMEI Tracing</li> </ul> <span>Its paid service and you need to purchase registration and get 24/7 customer support</span> </div> <h4><a href="https://play.google.com/store/apps/details?id=com.darkcoderz.datafinderService">Download DataFinder</a></h4></center> </body> </html>
内容的提问来源于stack exchange,提问作者Data Finder

