Java中Post请求分批发送大JSON数组，限制单次1000条或5MB

Got it, let's work through this batch sending problem together. The core idea is splitting your massive JSON array into chunks of ~1000 items (to fit the API's limits) and sending each chunk individually. Here's a practical, clean implementation using Jackson (a widely-used JSON library) to handle serialization, plus modified HTTP logic to manage batch requests.

Step 1: Add JSON Library Dependency #

First, if you're using Maven, add this to your pom.xml to include Jackson (it makes JSON serialization way less error-prone than manual string building):

<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.15.2</version> <!-- Use the latest stable version available -->
</dependency>

Step 2: Full Batch Sending Code #

Here's the modified code that splits your item list, serializes each batch to JSON, and sends it via POST:

import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.OutputStreamWriter;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.ArrayList;
import java.util.List;

public class BatchJsonSender {
    // Match the API's 1000-item limit (adjust if 5MB is hit before 1000 items)
    private static final int BATCH_SIZE = 1000;
    private static final ObjectMapper objectMapper = new ObjectMapper();

    public static void main(String args[]) throws Exception {
        // Replace this with your actual large list of items
        List<Item> largeItemList = generateSampleLargeList();

        // Split the big list into manageable batches
        List<List<Item>> batches = splitIntoBatches(largeItemList, BATCH_SIZE);

        // Send each batch to the API
        String apiUrl = "http://example.com";
        for (List<Item> batch : batches) {
            sendSingleBatch(apiUrl, batch);
            // Optional: Add a small delay to avoid overwhelming the API
            Thread.sleep(100);
        }
    }

    // Helper to split any list into batches of specified size
    private static <T> List<List<T>> splitIntoBatches(List<T> list, int batchSize) {
        List<List<T>> batches = new ArrayList<>();
        for (int i = 0; i < list.size(); i += batchSize) {
            int endIndex = Math.min(i + batchSize, list.size());
            batches.add(new ArrayList<>(list.subList(i, endIndex)));
        }
        return batches;
    }

    // Helper to send one batch via POST request
    private static void sendSingleBatch(String apiUrl, List<Item> batch) throws Exception {
        URL url = new URL(apiUrl);
        HttpURLConnection connection = (HttpURLConnection) url.openConnection();
        
        connection.setConnectTimeout(5000);
        connection.setReadTimeout(5000);
        connection.setRequestMethod("POST");
        connection.setDoOutput(true);
        connection.setRequestProperty("Content-Type", "application/json");

        // Serialize the batch to a JSON string matching your original structure
        String jsonBatch = objectMapper.writeValueAsString(new ItemsWrapper(batch));

        // Write the JSON to the request body (use try-with-resources to auto-close streams)
        try (OutputStreamWriter out = new OutputStreamWriter(connection.getOutputStream())) {
            out.write(jsonBatch);
            out.flush();
        }

        // Check the API response (add retry logic here if needed for failed requests)
        int responseCode = connection.getResponseCode();
        System.out.println("Batch sent successfully | Response Code: " + responseCode);
        
        connection.disconnect();
    }

    // POJO class matching your individual item JSON structure
    static class Item {
        private String Name;
        private String price;

        // Getters and setters are required for Jackson to serialize/deserialize
        public String getName() { return Name; }
        public void setName(String name) { Name = name; }
        public String getPrice() { return price; }
        public void setPrice(String price) { this.price = price; }
    }

    // Wrapper class to match your root "Items" JSON structure
    static class ItemsWrapper {
        private List<Item> Items;

        public ItemsWrapper(List<Item> items) { Items = items; }

        public List<Item> getItems() { return Items; }
        public void setItems(List<Item> items) { Items = items; }
    }

    // Helper to generate a sample large list for testing
    private static List<Item> generateSampleLargeList() {
        List<Item> items = new ArrayList<>();
        for (int i = 0; i < 5500; i++) {
            Item item = new Item();
            item.setName("Item " + i);
            item.setPrice(String.valueOf(i * 10));
            items.add(item);
        }
        return items;
    }
}

Key Details to Note #

• Batch Size Flexibility: The BATCH_SIZE is set to 1000, but if some items are extra large (and hit the 5MB limit before 1000 items), you can reduce this number or add a check to calculate batch size based on JSON byte size.
• Error Handling: The example includes basic response code checking, but you might want to add retry logic for 5xx errors or handle failed batches separately (e.g., log them for later reprocessing).
• Resource Management: Using try-with-resources for OutputStreamWriter ensures streams are properly closed, preventing resource leaks.

Original Reference Content #

Your original JSON structure:

Items:[ {"Name" : "Chair", "price" : "30"}, {"Name" : "Table", "price" : "40"}, {"Name" : "laptop", "price" : "300"}, ... ]

Your original Java snippet:

public static void main(String args[]) throws Exception{ 
    URL url = new URL("http://example.com"); 
    HttpURLConnection connection = (HttpURLConnection) url.openConnection(); 
    connection.setConnectTimeout(5000);//5 secs 
    connection.setReadTimeout(5000);//5 secs 
    connection.setRequestMethod("POST"); 
    connection.setDoOutput(true); 
    connection.setRequestProperty("Content-Type", "application/json"); 
    OutputStreamWriter out = new OutputStreamWriter(connection.getOutputStream()); 
    out.write("json object"); 
    out.flush(); 
    out.close(); 
}

内容的提问来源于stack exchange，提问作者user7183867