使用C#开发：能否将JSON/CSV加载至内存执行MongoDB查询？

Absolutely feasible! This is actually a super common scenario when you want to skip the hassle of spinning up a full MongoDB instance or importing data into a persistent collection just for ad-hoc queries. Let me walk you through two solid approaches to pull this off in C#.

Approach 1: Use an Embedded In-Memory MongoDB Instance #

If you want full support for MongoDB's native query language (including complex aggregations, indexes, and all the operators you're used to), an embedded MongoDB instance running in memory is the way to go. Tools like Mongo2Go or the official MongoDB.Embedded driver make this straightforward.

Step-by-Step Implementation #

1. First, install the relevant NuGet package. For Mongo2Go (great for testing and this use case):

   Install-Package Mongo2Go
   

2. Write code to start the embedded instance, load your data, and run queries:

   using MongoDB.Bson;
   using MongoDB.Driver;
   using Mongo2Go;
   using System.IO;
   
   // Define your data model to match your JSON/CSV structure
   public class User
   {
       public ObjectId Id { get; set; }
       public string Name { get; set; }
       public int Age { get; set; }
       public string Department { get; set; }
   }
   
   // Start an in-memory MongoDB instance
   var mongoRunner = MongoDbRunner.Start(storageEngine: "ephemeralForTest");
   var client = new MongoClient(mongoRunner.ConnectionString);
   var db = client.GetDatabase("InMemoryData");
   var userCollection = db.GetCollection<User>("Users");
   
   // Load JSON data and insert into the in-memory collection
   var jsonContent = File.ReadAllText("users.json");
   var users = BsonSerializer.Deserialize<List<User>>(jsonContent);
   userCollection.InsertMany(users);
   
   // Run native MongoDB queries just like you would on a real instance
   // Example 1: Filter users over 30
   var usersOver30 = userCollection.Find(u => u.Age > 30).ToList();
   
   // Example 2: Aggregate to count users per department
   var deptCounts = userCollection.Aggregate()
       .Group(u => u.Department, g => new { Department = g.Key, Count = g.Count() })
       .ToList();
   
   // Clean up when done
   mongoRunner.Dispose();
   

Approach 2: Query In-Memory Objects Directly with MongoDB LINQ #

If you don't need the full MongoDB feature set and just want to run simple to moderately complex queries, you can load your JSON/CSV into a C# object list and use MongoDB's LINQ extensions to query the in-memory data. This is lighter weight since you don't need to spin up an embedded database.

Step-by-Step Implementation #

1. Make sure you have the MongoDB .NET driver installed:

   Install-Package MongoDB.Driver
   

2. Load your data and query it with LINQ:

   using MongoDB.Driver.Linq;
   using MongoDB.Bson;
   using System.IO;
   using System.Linq;
   
   // Reuse the same User model from Approach 1
   public class User { /* ... */ }
   
   // Load JSON into an in-memory list
   var jsonContent = File.ReadAllText("users.json");
   var users = BsonSerializer.Deserialize<List<User>>(jsonContent);
   
   // Convert the list to a queryable and run MongoDB-style LINQ queries
   var queryableUsers = users.AsQueryable();
   
   // Example 1: Filter active users (assuming a Status property)
   var activeUsers = queryableUsers.Where(u => u.Status == "Active").ToList();
   
   // Example 2: Sum salaries per department
   var deptSalaryTotals = queryableUsers
       .GroupBy(u => u.Department)
       .Select(g => new { Department = g.Key, TotalSalary = g.Sum(u => u.Salary) })
       .ToList();
   

Handling CSV Files #

For CSV data, you'll first need to parse the CSV into C# objects. The CsvHelper library is perfect for this:

Install-Package CsvHelper

Then parse and query:

using CsvHelper;
using System.Globalization;
using System.IO;

// Parse CSV into a list of User objects
List<User> csvUsers;
using (var reader = new StreamReader("users.csv"))
using (var csv = new CsvReader(reader, CultureInfo.InvariantCulture))
{
    csvUsers = csv.GetRecords<User>().ToList();
}

// Now use either Approach 1 or 2 to query csvUsers

Key Notes #

• Embedded Instance Pros: Full MongoDB query support, including aggregations like $lookup, $unwind, and indexes. Great for replicating production query logic in memory.
• LINQ Approach Pros: No database instance required, faster for simple queries, lower overhead.
• Limitations: The LINQ approach doesn't support all MongoDB operators, so for complex queries, stick with the embedded instance.

内容的提问来源于stack exchange，提问作者Mhmd