基于ASP.NET MVC的客户端内存存储查询大数据方案求助

Hey there! Let's break down feasible solutions for offloading data storage and querying to the client side in your ASP.NET MVC project—no server memory bloat required. Here are the approaches I've seen work well in similar real-world scenarios:

The simplest approach is to load your dataset once (or in chunks) from your ASP.NET MVC backend into a JavaScript object, then handle all queries directly on the client. This avoids tying up server memory entirely, as the data lives only in the user's browser session.

Example Implementation: #

First, fetch data from your MVC action:

// Store data in a module-scoped variable (avoid polluting global scope)
let clientSideDataset = [];

// Fetch data on page load
async function initializeDataset() {
    const response = await fetch('/YourController/GetFullDataset');
    clientSideDataset = await response.json();
}

// Query function example (filter by a specific property)
function filterDataset(filterValue) {
    return clientSideDataset.filter(item => item.TargetProperty === filterValue);
}

Pros: No extra dependencies, fast querying once data is loaded.
Cons: Data is lost on page refresh; not ideal for extremely large datasets (since it eats into the browser's available memory).

If you need to persist data across page navigations (within a session or long-term), use the browser's built-in storage APIs. These let you store stringified JSON data directly on the client.

Example with SessionStorage (Session-Only Persistence): #

// Save data to session storage
function persistDataset(data) {
    sessionStorage.setItem('appDataset', JSON.stringify(data));
}

// Load data from storage
function loadPersistedDataset() {
    const storedData = sessionStorage.getItem('appDataset');
    return storedData ? JSON.parse(storedData) : [];
}

// Reuse the same query functions as the in-memory approach

Notes:

• SessionStorage limits are ~5MB per domain, LocalStorage has similar limits but persists across browser restarts.
• Never store sensitive data here unless you encrypt it first—storage is plaintext and accessible via the browser's dev tools.

For datasets bigger than 5MB, IndexedDB is the way to go. It's a browser-based asynchronous database that supports indexing, making complex queries fast even with large volumes of data. To simplify development, use a wrapper library like Dexie.js instead of writing raw IndexedDB code.

Example with Dexie.js: #

// Initialize the database
const db = new Dexie('ProjectDatasetDB');
db.version(1).stores({
    records: 'id, Category, CreatedDate' // Define indexed fields for faster queries
});

// Bulk import data from your MVC backend
async function importDataset(data) {
    await db.records.bulkAdd(data);
}

// Query example (filter by category)
async function getRecordsByCategory(category) {
    return await db.records.where('Category').equals(category).toArray();
}

Pros: Supports large datasets (tens/hundreds of MB), asynchronous operations don't block the UI, indexes speed up queries.
Cons: Slightly steeper learning curve compared to simpler storage options.

If your data is too big to load entirely into the client (even with IndexedDB), combine server-side pagination with client-side virtual scrolling. Your ASP.NET MVC backend returns only small chunks of data at a time, and the frontend renders only the rows visible in the user's viewport—keeping memory usage low.

Backend ASP.NET MVC Action: #

public JsonResult GetPaginatedData(int pageNumber, int pageSize)
{
    // Use Entity Framework (or your ORM) to fetch paginated data
    var paginatedRecords = _dbContext.LargeDataset
        .Skip((pageNumber - 1) * pageSize)
        .Take(pageSize)
        .ToList();
    
    return Json(paginatedRecords, JsonRequestBehavior.AllowGet);
}

On the frontend, use a virtual scrolling component (or build a simple one) to load pages as the user scrolls, so only a small subset of data is kept in memory at any time.

• Data Sensitivity: Avoid storing sensitive data in client-side storage unless you encrypt it first.
• Initial Load Time: Loading large datasets upfront can slow down page load—consider lazy loading chunks or showing a loading spinner.
• Browser Compatibility: All modern browsers support these APIs, but if you need to target older versions, add fallbacks (e.g., use SessionStorage if IndexedDB isn't available).

内容的提问来源于stack exchange，提问作者Sathish Ram