无需SQL/MySQL/JTable构建文本表格及AWT处理大CSV数据性能优化咨询

First, let’s break down the proper way to handle text file data without relying on databases or JTable:

1. Model Your Data #

Instead of juggling raw strings, create a plain Java object (POJO) to represent each record. This makes data manipulation and access way cleaner. For example, if your CSV holds user data:

public class User {
    private String name;
    private int age;
    private String email;

    // Constructor, getters, setters
    public User(String name, int age, String email) {
        this.name = name;
        this.age = age;
        this.email = email;
    }

    public String getName() { return name; }
    public int getAge() { return age; }
    public String getEmail() { return email; }
}

2. Efficiently Read & Parse the Text/CSV File #

Use BufferedReader for fast line-by-line reading—it’s significantly quicker than Scanner for large files. For CSV, implement a robust parser that handles edge cases like quoted fields containing commas:

import java.io.BufferedReader;
import java.io.FileReader;
import java.util.ArrayList;
import java.util.List;

public class CsvReader {
    public static List<User> readUsers(String filePath) throws Exception {
        List<User> users = new ArrayList<>();
        try (BufferedReader br = new BufferedReader(new FileReader(filePath))) {
            String line;
            br.readLine(); // Skip header row if present
            while ((line = br.readLine()) != null) {
                String[] fields = parseCsvLine(line);
                String name = fields[0];
                int age = Integer.parseInt(fields[1]);
                String email = fields[2];
                users.add(new User(name, age, email));
            }
        }
        return users;
    }

    // Helper to handle quoted CSV fields
    private static String[] parseCsvLine(String line) {
        List<String> fields = new ArrayList<>();
        StringBuilder currentField = new StringBuilder();
        boolean inQuotes = false;

        for (char c : line.toCharArray()) {
            if (c == '"') {
                inQuotes = !inQuotes;
            } else if (c == ',' && !inQuotes) {
                fields.add(currentField.toString().trim());
                currentField.setLength(0);
            } else {
                currentField.append(c);
            }
        }
        fields.add(currentField.toString().trim());
        return fields.toArray(new String[0]);
    }
}

3. Optimize Data Storage #

Use ArrayList<User> for most scenarios—it offers fast random access, perfect for table views. If you need frequent insertions/deletions at arbitrary positions, LinkedList might be better, but ArrayList is more memory-efficient for large datasets.

The main issue with your current AWT setup is probably rendering all rows at once, which clogs the UI thread and wastes memory. The fix is virtual scrolling—only render the rows that are visible in the viewport. Here’s how to implement it:

Key Concepts #

• Track scroll position to determine which rows are visible.
• Calculate the total height of all rows so the scroll pane knows the full scroll range.
• Only paint visible rows when the component repaints.

Example Implementation #

import java.awt.*;
import java.awt.event.AdjustmentEvent;
import java.awt.event.AdjustmentListener;
import java.util.List;

public class VirtualTable extends Panel {
    private List<User> data;
    private int rowHeight = 25; // Height of each row
    private int visibleRows;
    private int topRowIndex = 0; // First visible row index

    public VirtualTable(List<User> data) {
        this.data = data;
        setLayout(null); // Manual positioning
    }

    @Override
    public void paint(Graphics g) {
        super.paint(g);
        visibleRows = getHeight() / rowHeight;
        int endRow = Math.min(topRowIndex + visibleRows, data.size());

        // Draw header
        g.setColor(Color.LIGHT_GRAY);
        g.fillRect(0, 0, getWidth(), rowHeight);
        g.setColor(Color.BLACK);
        g.drawString("Name", 10, 15);
        g.drawString("Age", 150, 15);
        g.drawString("Email", 200, 15);

        // Draw visible rows
        for (int i = topRowIndex; i < endRow; i++) {
            User user = data.get(i);
            int y = (i - topRowIndex) * rowHeight + rowHeight;
            // Alternate row colors for readability
            g.setColor(i % 2 == 0 ? Color.WHITE : new Color(240, 240, 240));
            g.fillRect(0, y, getWidth(), rowHeight);
            g.setColor(Color.BLACK);
            g.drawString(user.getName(), 10, y + 15);
            g.drawString(String.valueOf(user.getAge()), 150, y + 15);
            g.drawString(user.getEmail(), 200, y + 15);
        }
    }

    // Update visible rows when scrolling
    public void setTopRowIndex(int index) {
        this.topRowIndex = Math.max(0, Math.min(index, data.size() - visibleRows));
        repaint();
    }

    // Total height of all rows (including header)
    public int getTotalHeight() {
        return data.size() * rowHeight + rowHeight;
    }

    public static void main(String[] args) throws Exception {
        List<User> users = CsvReader.readUsers("large_data.csv");

        Frame frame = new Frame("Virtual CSV Table");
        frame.setSize(800, 600);
        frame.setLayout(new BorderLayout());

        VirtualTable table = new VirtualTable(users);
        ScrollPane scrollPane = new ScrollPane(ScrollPane.SCROLLBARS_ALWAYS);
        scrollPane.add(table);

        // Link scroll position to visible rows
        scrollPane.getVAdjustable().addAdjustmentListener(e -> {
            int scrollPos = e.getValue();
            table.setTopRowIndex(scrollPos / table.rowHeight);
        });

        // Configure scroll range
        scrollPane.getVAdjustable().setMaximum(table.getTotalHeight());

        frame.add(scrollPane, BorderLayout.CENTER);
        frame.setVisible(true);
        frame.addWindowListener(new java.awt.event.WindowAdapter() {
            public void windowClosing(java.awt.event.WindowEvent e) {
                System.exit(0);
            }
        });
    }
}

Why This Works #

• Reduced Rendering Load: Instead of painting 10k rows, we only render the 20-30 that fit in the viewport.
• Memory Efficiency: No thousands of UI components (like Labels) — we draw text directly onto the panel.
• Smooth Scrolling: The scroll listener updates the visible row index in real-time, triggering only necessary repaints.

Additional Tips:

• Background Loading: Always read CSV files in a background thread (not the UI thread) to avoid freezing the app during load. Use java.util.concurrent for this.
• On-Demand Chunking: For extremely large files (100k+ rows), load data in chunks as the user scrolls instead of all at once.
• Double Buffering: Enable double buffering on your panel to reduce flicker during repaints.

内容的提问来源于stack exchange，提问作者mightguy