如何将SQL、Excel数据集导入Parse Server？15万条数据需批量导入

Hey there! Dealing with 150k+ records one by one is totally a slog—let’s get you set up with efficient batch imports for both SQL and Excel datasets into Parse Server. Here’s how to tackle each scenario:

Since direct SQL-to-Parse imports aren’t built-in, we’ll use a middle step (CSV or MongoDB JSON) to bridge the gap, which is way faster than manual entry.

Step 1: Export SQL Data to CSV #

Nearly every SQL database (MySQL, PostgreSQL, SQL Server, etc.) lets you export tables to CSV. Make sure:

• CSV column names exactly match your Parse Server class field names (Parse is case-sensitive!)
• For nested fields (like user.profile.name), use dot notation in your CSV headers
• Clean up any invalid characters or missing values that might cause parsing errors

Step 2: Batch Import via Node.js Script #

Parse’s Batch API lets you create up to 500 records per request—perfect for large datasets. Here’s a ready-to-use script:

First, install dependencies:

npm install parse csv-parser

Then create a script (e.g., import.js):

const Parse = require('parse/node');
const fs = require('fs');
const csv = require('csv-parser');

// Configure Parse connection
Parse.initialize('YOUR_APP_ID', 'YOUR_JAVASCRIPT_KEY');
Parse.serverURL = 'http://your-parse-server-url/parse';

const records = [];
const batchSize = 500; // Parse's maximum allowed per batch

// Read CSV file into memory
fs.createReadStream('your-sql-data.csv')
  .pipe(csv())
  .on('data', (data) => records.push(data))
  .on('end', async () => {
    console.log(`Total records to import: ${records.length}`);
    
    // Process in batches
    for (let i = 0; i < records.length; i += batchSize) {
      const batch = records.slice(i, i + batchSize);
      const parseObjects = batch.map(record => {
        const obj = new Parse.Object('YourParseClassName');
        // Map CSV columns to Parse fields
        for (const [key, value] of Object.entries(record)) {
          // Convert data types if needed (e.g., strings to numbers/dates)
          obj.set(key, value);
        }
        return obj;
      });

      try {
        await Parse.Object.saveAll(parseObjects);
        console.log(`✅ Batch ${Math.floor(i/batchSize) + 1} imported successfully`);
      } catch (err) {
        console.error(`❌ Error importing batch ${Math.floor(i/batchSize) + 1}:`, err.message);
        // Add retry logic here if needed for failed batches
      }
    }
    console.log('🎉 All records imported!');
  });

Run the script with:

node import.js

Alternative: Direct MongoDB Import (For Advanced Users) #

If your Parse Server uses MongoDB as its backend, you can skip the CSV step entirely:

1. Export your SQL data to MongoDB-compatible JSON (most SQL tools support this, or use a conversion script)
2. Use the mongoimport command to push the JSON directly to your Parse database:

mongoimport --uri "mongodb://your-mongo-host:27017/your-parse-db" --collection "_YourParseClassName" --file your-sql-data.json --jsonArray

Note: Parse prepends an underscore to custom class names (e.g., Product becomes _Product in MongoDB)

Excel files need a quick conversion to CSV first, then you can reuse the same batch methods above.

Step 1: Convert Excel to CSV #

Option 1: Manual Conversion #

Open your Excel file, go to File > Save As, and select "CSV (Comma delimited)" as the format. For multi-sheet files, repeat this for each sheet.

Option 2: Bulk Conversion with Python #

If you have multiple sheets or large Excel files, use pandas to automate conversion:

import pandas as pd

excel_file = pd.ExcelFile('your-excel-file.xlsx')
for sheet_name in excel_file.sheet_names:
    df = excel_file.parse(sheet_name)
    # Clean data if needed (e.g., remove empty rows)
    df = df.dropna(how='all')
    df.to_csv(f'{sheet_name}.csv', index=False)

Step 2: Import CSV to Parse #

Once you have your CSV(s), use the same Node.js batch script from the SQL section—just swap out the CSV file path and Parse class name.

• Test First: Run your script with a small batch (10-100 records) to verify field mapping and data types before importing everything.
• Add Retries: For flaky connections, add retry logic to your script for failed batches (use a library like p-retry if needed).
• Index After Import: After all records are in, add indexes on frequently queried fields in Parse Server—this will speed up future database queries.
• Monitor Server Load: If your Parse Server is hosted on a small instance, consider scaling up temporarily during the import to avoid timeouts.

内容的提问来源于stack exchange，提问作者Furkan Topal