Alright, let's break down how you can scan that /images/ directory to uncover all images—both the ones you know and the unknown ones. Here are the most practical approaches, ordered by ease of use and common utility:

The simplest first step: just visit https://www.somepage.com/images/ directly. Many web servers (like Apache or Nginx) are configured to show a list of all files in a directory if there's no index.html (or similar) file present. If this works, you'll see every image in the directory laid out for you—no tools needed.

If directory listing is disabled, tools like Gobuster or Dirb are designed to guess filenames by testing common patterns against the directory. Given your known images follow a [animal]_[number].jpg pattern, you can tailor your approach for better results.

For example, using Gobuster:

gobuster dir -u https://www.somepage.com/images/ -w /path/to/custom-wordlist.txt -x jpg,png,gif

• -w: Points to a wordlist. You can create a custom one that includes animal names paired with number ranges (e.g., cat_1 to cat_1000, dog_1 to dog_1000) plus generic image filenames like photo_1, img_50 to cover other possible patterns.
• -x: Specifies which file extensions to target (stick to common image formats).

If you prefer a more hands-on approach, you can build a basic crawler to scrape all image links from the entire website. This works because any image used on the site will be referenced in an <img> tag somewhere.

Here's a quick Python example using requests and BeautifulSoup:

import requests
from bs4 import BeautifulSoup
from urllib.parse import urljoin

base_url = "https://www.somepage.com/"
target_dir = "/images/"
visited_pages = set()
found_images = set()

def crawl_page(url):
    if url in visited_pages:
        return
    visited_pages.add(url)
    try:
        response = requests.get(url, timeout=10)
        response.raise_for_status()
        soup = BeautifulSoup(response.text, "html.parser")
        
        # Collect all images pointing to the /images/ directory
        for img_tag in soup.find_all("img"):
            img_src = img_tag.get("src")
            if not img_src:
                continue
            full_img_url = urljoin(base_url, img_src)
            if target_dir in full_img_url:
                found_images.add(full_img_url)
        
        # Recursively crawl all internal links
        for link_tag in soup.find_all("a"):
            link_href = link_tag.get("href")
            if not link_href:
                continue
            full_link_url = urljoin(base_url, link_href)
            if base_url in full_link_url:  # Only crawl internal pages
                crawl_page(full_link_url)
    except Exception as e:
        print(f"Failed to crawl {url}: {str(e)}")

# Start crawling from the homepage
crawl_page(base_url)

# Print all found images
print("Discovered images in /images/:")
for img in found_images:
    print(img)

This script will traverse every page on the site, extract all image URLs, and filter out only those that live in the /images/ directory.

Don't overlook these quick checks:

• Visit https://www.somepage.com/sitemap.xml: Some sites include all their media files in the sitemap, which would give you a direct list of images.
• Check https://www.somepage.com/robots.txt: This file might tell you if crawling the /images/ directory is allowed, and sometimes even hint at existing files or subdirectories.

Critical Notes Before You Start #

• Legality & Ethics: Always ensure you have explicit permission to scan or crawl the website. Unauthorized scraping or directory brute-forcing may violate the site's Terms of Service or local laws. Check robots.txt first, and if in doubt, reach out to the site owner.
• Rate Limiting: Whether using a tool or a custom crawler, slow down your request rate. Bombarding the server with too many requests too quickly can get your IP blocked, and it's respectful to avoid straining the site's resources.
• Customize Your Wordlist: Since you know the existing images use animal_number.jpg, prioritize generating a wordlist that mirrors this pattern—it'll drastically improve your brute-force efficiency compared to generic lists.

内容的提问来源于stack exchange，提问作者Snochacz