Flutter技术问题：如何抓取HTML页面指定元素（如商品价格）

Hey there! Let's work through this HTML scraping issue for your animal shelter app—such a meaningful project, kudos for building it!

To grab that real-time dog food price from a website and pair it with your local image, here's a step-by-step approach using Dart/Flutter tools:

1. Add Required Dependencies #

First, you'll need two packages to handle HTTP requests and HTML parsing. Add these to your pubspec.yaml:

dependencies:
  flutter:
    sdk: flutter
  http: ^1.1.0
  html: ^0.15.4

Run flutter pub get to install them.

2. Write the Price Scraping Function #

This function will fetch the webpage, parse the HTML, and extract the price element you need. You'll need to adjust the CSS selector to match the actual element on your target website (more on that below).

import 'package:http/http.dart' as http;
import 'package:html/parser.dart' as parser;
import 'package:html/dom.dart' as dom;

Future<String?> fetchDogFoodPrice(String targetUrl) async {
  try {
    // Send a request with a browser-like user agent to avoid being blocked
    final response = await http.get(
      Uri.parse(targetUrl),
      headers: {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36'
      },
    );

    if (response.statusCode == 200) {
      // Parse the raw HTML into a document we can query
      final document = parser.parse(response.body);
      
      // Replace this selector with the one matching your target price element
      // Example: If the price is in a <span class="product-price">$19.99</span>
      final priceElement = document.querySelector('.product-price');
      
      if (priceElement != null) {
        // Clean up the text (remove extra spaces, currency symbols if needed)
        String rawPrice = priceElement.text.trim();
        String cleanedPrice = rawPrice.replaceAll(RegExp(r'[^0-9.]'), '');
        return cleanedPrice;
      } else {
        print("Couldn't find the price element on the page");
        return null;
      }
    } else {
      print("Failed to load page: Status code ${response.statusCode}");
      return null;
    }
  } catch (e) {
    print("Error fetching price: $e");
    return null;
  }
}

3. Integrate with Your Flutter Donation Page #

Now, pair this function with your local image and display the price in your UI:

class DonationPage extends StatefulWidget {
  @override
  _DonationPageState createState() => _DonationPageState();
}

class _DonationPageState extends State<DonationPage> {
  String? _dogFoodPrice;
  bool _isLoading = false;

  @override
  void initState() {
    super.initState();
    _loadPrice();
  }

  Future<void> _loadPrice() async {
    setState(() => _isLoading = true);
    // Replace with your actual product page URL
    final price = await fetchDogFoodPrice('https://example.com/dog-food-product');
    setState(() {
      _dogFoodPrice = price;
      _isLoading = false;
    });
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: const Text('Donate Dog Food')),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: [
            // Your local dog food image (update the asset path)
            Image.asset('assets/images/dog_food.png', width: 200, height: 200),
            const SizedBox(height: 20),
            if (_isLoading)
              const CircularProgressIndicator()
            else if (_dogFoodPrice != null)
              Text(
                'Current Price: \$$_dogFoodPrice',
                style: const TextStyle(fontSize: 20, fontWeight: FontWeight.bold),
              )
            else
              const Text('Failed to load price'),
            const SizedBox(height: 20),
            ElevatedButton(
              onPressed: _loadPrice,
              child: const Text('Refresh Price'),
            )
          ],
        ),
      ),
    );
  }
}

Key Tips to Troubleshoot #

• Find the Right CSS Selector: Go to your target website, right-click the price element, select "Inspect", then copy the element's class, ID, or CSS path. Use that in querySelector().
• Dynamic Content?: If the price loads via JavaScript (not in the initial HTML), the above method won't work. Instead, use the webview_flutter package to load the page and run JavaScript to extract the price:

  // Example snippet for webview approach
  WebView(
    initialUrl: 'https://example.com/dog-food-product',
    javascriptMode: JavascriptMode.unrestricted,
    onPageFinished: (url) {
      controller.runJavascriptReturningResult(
        "document.querySelector('.product-price').textContent"
      ).then((result) {
        setState(() {
          _dogFoodPrice = result.replaceAll('"', '').trim();
        });
      });
    },
  )
  

• Compliance: Always check the website's robots.txt and terms of service to make sure scraping is allowed.
• Caching: Add a cache layer (e.g., using shared_preferences) to store the price for 1-2 hours, so you don't spam the website with requests.

内容的提问来源于stack exchange，提问作者Zoltán Györkei