如何修改PHP爬虫脚本避免simple_html_dom的plaintext空对象错误?
解决simple_html_dom爬取时"Attempt to read property 'plaintext' on null"错误
在使用simple_html_dom.php爬取网页时,不同页面的DOM元素数量不一致,当目标元素不存在时直接读取plaintext属性会触发"Attempt to read property 'plaintext' on null"错误。以下是修改后的PHP脚本,实现元素不存在时跳过读取操作,避免报错:
<?php include 'simple_html_dom.php'; include '../lib/_config.php'; // URL to scrape $url = "https://m.spareroom.com/rooms-for-rent/ca/sacramento_county/rancho_cordova/102552139"; //$url = "https://m.spareroom.com/rooms-for-rent/ca/sacramento_county/rancho_cordova/102553449"; // Get the HTML content $html = file_get_html($url); // 标题部分:foreach本身安全,找不到元素不会执行 foreach($html->find('h1.heading') as $title) echo $title->innertext . '<br>'; ################################################################################ // 价格部分:修复语法错误,先判断元素存在再处理 foreach($html->find('span.key-features__price') as $price) { $clean_price = clean_price($price); echo $clean_price . '<br>'; } // 位置特征:foreach安全 foreach($html->find('li.key-features__feature') as $location) echo $location->plaintext. '<br>'; echo "<h3> 描述 </h3>"; // 描述部分:foreach安全 foreach($html->find('div.feature__description-body') as $description) echo $description->plaintext . "<br>"; ####### 可用性 ########################################################### echo "<h3> 可用性 </h3>"; // 单个元素先判断存在再读取 $availableEl = $html->find('dd.feature-list__value', 0); if ($availableEl) { echo "Available : " . $availableEl->plaintext . "<br>"; } $minTermEl = $html->find('dd.feature-list__value', 1); if ($minTermEl) { echo "Minimum term : " . $minTermEl->plaintext . "<br>"; } $maxTermEl = $html->find('dd.feature-list__value', 2); if ($maxTermEl) { echo "Maximum term : " . $maxTermEl->plaintext . "<br>"; } ###### 额外费用 ############################################################## echo "<h3> 额外费用 </h3>"; $depositEl = $html->find('dd.feature-list__value', 3); if ($depositEl) { echo "Deposit : " . $depositEl->plaintext . "<br>"; } $utilitiesEl = $html->find('dd.feature-list__value', 4); if ($utilitiesEl) { echo "Utilities included? : " . $utilitiesEl->plaintext . "<br>"; } $feesApplyEl = $html->find('dd.feature-list__value', 5); if ($feesApplyEl) { echo "Fees apply? : " . $feesApplyEl->plaintext . "<br>"; } #### 设施 ################################################################# echo "<h3> 设施 </h3>"; $furnishingsEl = $html->find('dd.feature-list__value', 6); if ($furnishingsEl) { echo "Furnishings : " . $furnishingsEl->plaintext . "<br>"; } $parkingEl = $html->find('dd.feature-list__value', 7); if ($parkingEl) { echo "Parking : " . $parkingEl->plaintext . "<br>"; } $garageEl = $html->find('dd.feature-list__value', 8); if ($garageEl) { echo "Garage : " . $garageEl->plaintext . '<br>'; } $gardenTerraceEl = $html->find('dd.feature-list__value', 9); if ($gardenTerraceEl) { echo "Garden/terrace : " . $gardenTerraceEl->plaintext . '<br>'; } $balconyPatioEl = $html->find('dd.feature-list__value', 10); if ($balconyPatioEl) { echo "Balcony/patio : " . $balconyPatioEl->plaintext . "<br>"; } $disabledAccessEl = $html->find('dd.feature-list__value', 11); if ($disabledAccessEl) { echo "Disabled access : " . $disabledAccessEl->plaintext . "<br>"; } $livingRoomEl = $html->find('dd.feature-list__value', 12); if ($livingRoomEl) { echo "Living room : " . $livingRoomEl->plaintext . "<br>"; } $broadbandEl = $html->find('dd.feature-list__value', 13); if ($broadbandEl) { echo "Broadband : " . $broadbandEl->plaintext . "<br>"; } ################################################################################ #### 当前住户情况 ######################################################### echo "<h3> 当前住户情况 </h3>"; $roommatesEl = $html->find('dd.feature-list__value', 14); if ($roommatesEl) { echo "Number Of Roommates : " . $roommatesEl->plaintext . "<br>"; } $totalRoomsEl = $html->find('dd.feature-list__value', 15); if ($totalRoomsEl) { echo "Total # rooms : " . $totalRoomsEl->plaintext . "<br>"; } else { echo "Total # rooms : 信息未提供<br>"; } $agesEl = $html->find('dd.feature-list__value', 16); if ($agesEl) { echo "Ages : " . $agesEl->plaintext . "<br>"; } $smokingOkEl = $html->find('dd.feature-list__value', 17); if ($smokingOkEl) { echo "Smoking Ok : " . $smokingOkEl->plaintext . "<br>"; } $petsOkEl = $html->find('dd.feature-list__value', 18); if ($petsOkEl) { echo "Pets Ok : " . $petsOkEl->plaintext . "<br>"; } $languageEl = $html->find('dd.feature-list__value', 19); if ($languageEl) { echo "Language : " . $languageEl->plaintext . "<br>"; } $nationalityEl = $html->find('dd.feature-list__value', 20); if ($nationalityEl) { echo "Nationality : " . $nationalityEl->plaintext . "<br>"; } $roommateOccupationEl = $html->find('dd.feature-list__value', 21); if ($roommateOccupationEl) { echo "Roommate Occupation : " . $roommateOccupationEl->plaintext . "<br>"; } $interestEl = $html->find('dd.feature-list__value', 22); if ($interestEl) { echo "Interests : " . $interestEl->plaintext . "<br>"; } $houseGenderEl = $html->find('dd.feature-list__value', 23); if ($houseGenderEl) { echo "Gender : " . $houseGenderEl->plaintext . "<br>"; } #### 新室友偏好 ################################################## echo "<h3> 新室友偏好 </h3>"; $couplesOkEl = $html->find('dd.feature-list__value', 24); if ($couplesOkEl) { echo "Couples ok? " . $couplesOkEl->plaintext . "<br>"; } $smokingOkPrefEl = $html->find('dd.feature-list__value', 25); if ($smokingOkPrefEl) { echo "Smoking ok? " . $smokingOkPrefEl->plaintext . "<br>"; } $petsOkPrefEl = $html->find('dd.feature-list__value', 26); if ($petsOkPrefEl) { echo "Pets ok? " . $petsOkPrefEl->plaintext . "<br>"; } $occupationPrefEl = $html->find('dd.feature-list__value', 27); if ($occupationPrefEl) { echo "Occupation? " . $occupationPrefEl->plaintext . "<br>"; } $referencesEl = $html->find('dd.feature-list__value', 28); if ($referencesEl) { echo "References? " . $referencesEl->plaintext . "<br>"; } $minAgeEl = $html->find('dd.feature-list__value', 29); if ($minAgeEl) { echo "Min age " . $minAgeEl->plaintext . "<br>"; } $maxAgeEl = $html->find('dd.feature-list__value', 30); if ($maxAgeEl) { echo "Max age " . $maxAgeEl->plaintext . "<br>"; } $genderPreferredEl = $html->find('dd.feature-list__value', 31); if ($genderPreferredEl) { echo "Gender Prefered " . $genderPreferredEl->plaintext . "<br>"; } #### 用户资料名称 ################################################## echo "<h3> 用户资料名称 </h3>"; foreach($html->find('strong.profile-photo__name') as $user_profile) echo $user_profile->plaintext . '<br>'; foreach($html->find('ul.thumbs img') as $image_one) echo $image_one->src . '<br>'; ################################################################################ ?>
修改要点
- 单个元素安全检查:所有通过
find(selector, index)获取的单个元素,先判断元素是否存在(if ($element)),再读取plaintext属性,避免null报错; - 修复语法错误:价格部分的foreach循环缺少大括号,导致逻辑错误,已补充;
- 处理缺失元素的提示:对部分缺失的元素(如总房间数),输出友好的默认提示,而不是输出未定义变量;
- foreach循环安全性:foreach遍历
find()结果时,若找不到元素,循环体不会执行,无需额外判断。
内容的提问来源于stack exchange,提问作者TrevTech
相关产品推荐
相关产品推荐

