数据爬取时如何去除换行符、无关文本并逐行输出名言

Hey there! I see your scraper is pulling in a ton of extra newlines, sidebar content, and irrelevant text—let's fix that so you get just the clean, line-by-line quotes from H. Jackson Brown Jr. Here's a step-by-step solution:

First, Let's Diagnose the Original Code Issues #

Your current selector ($('.clearfix').parent().children()) is way too broad—it's grabbing entire sections of the page (like related authors, topics, and site navigation) instead of targeting just the quote elements. Plus, you're not cleaning up the raw text, which is why you're getting all those \n characters and messy whitespace.

Optimized Code Solution #

Here's a revised version that targets the right elements and cleans up the output:

const request = require('request');
const cheerio = require('cheerio');
const fs = require("fs");

const url = "https://www.brainyquote.com/authors/h_jackson_brown_jr";

request(url, function(err, response, html) {
  if (!err && response.statusCode === 200) {
    const $ = cheerio.load(html);
    const quotes = [];

    // Target only the quote containers on the page
    $('.bqQt').each(function() {
      // Extract the quote text and clean it up
      const quoteText = $(this).find('.b-qt').text()
        .replace(/\s+/g, ' ') // Replace multiple spaces/newlines with a single space
        .trim(); // Remove leading/trailing whitespace
      
      // Extract the author name and clean it
      const author = $(this).find('.bq-aut').text()
        .replace(/\s+/g, ' ')
        .trim();

      // Only add valid quotes (skip empty entries)
      if (quoteText && author) {
        quotes.push(`${quoteText} — ${author}`);
      }
    });

    // Print quotes line-by-line to console
    console.log("Cleaned Quotes:\n");
    quotes.forEach(quote => console.log(quote));

    // Write to file with each quote on a new line
    fs.writeFile("clean_quotes.txt", quotes.join('\n'), (writeErr) => {
      if (writeErr) {
        console.error("Error writing file:", writeErr);
      } else {
        console.log("\nQuotes saved to clean_quotes.txt!");
      }
    });
  } else {
    console.error("Error fetching page:", err || `Status code: ${response.statusCode}`);
  }
});

Key Improvements Explained #

• Precise Element Targeting: Using .bqQt (the container for each quote entry) ensures we only grab quote-related content, not sidebar or navigation sections.
• Text Cleaning: The .replace(/\s+/g, ' ') regex replaces all consecutive whitespace (newlines, tabs, multiple spaces) with a single space, and .trim() removes any leftover leading/trailing whitespace.
• Structured Output: We combine the quote and author into a readable format, then output each quote on its own line in both the console and the saved file.
• Error Handling: Added checks for HTTP status codes and file-writing errors to make the script more robust.

Example Clean Output #

Love is when the other person's happiness is more important than your own. — H. Jackson Brown, Jr.
The best preparation for tomorrow is doing your best today. — H. Jackson Brown, Jr.
Sometimes the heart sees what is invisible to the eye. — H. Jackson Brown, Jr.
Remember that the most valuable antiques are dear old friends. — H. Jackson Brown, Jr.
Our character is what we do when we think no one is looking. — H. Jackson Brown, Jr.

内容的提问来源于stack exchange，提问作者Sunil kumar