ASP.NET MVC 5中Solr数据索引方案选择：DIH还是SolrNet？

Hey there! Let's break down your two options for indexing event data from two databases into Solr in your ASP.NET MVC 5 app, and help you pick the best fit based on your needs.

This is Solr's built-in tool for pulling data directly from databases into indexes, no extra application code required.

Pros #

• Low code overhead: All configuration-driven, so you don't have to write C# code to fetch and push data.
• Built-in incremental sync: Supports delta imports to only update changed records, perfect for scheduled batch syncs.
• Multi-data-source support: You can configure connections to both your databases in a single DIH setup, making it easy to pull data from both.

Cons #

• Limited flexibility: Complex data transformations or business logic (like aggregating data across databases) are hard to implement—you'd need custom DIH plugins or rely on limited built-in converters.
• Debugging challenges: Troubleshooting sync issues is trickier compared to debugging application code, since you're working with Solr config files.
• Tight coupling: Changes to your database schema will require updates to your DIH configuration.

This approach uses the SolrNet library to fetch data from your databases in your ASP.NET code, transform it as needed, and push it to Solr.

Pros #

• Full control: You own the entire pipeline—data fetching, transformation, and indexing. Great for complex logic like merging related data from both databases or adding business-specific fields.
• Real-time indexing: You can trigger indexing immediately when an event is created/updated in your app, ensuring Solr has the latest data.
• Easy debugging: Use standard C# debugging tools to troubleshoot data fetching or transformation issues.
• Seamless integration: Fits naturally with your existing MVC app's architecture, using your existing database contexts (like Entity Framework) to pull data.

Cons #

• More code to maintain: You'll need to write and maintain code to fetch data from both databases, handle incremental syncs, and manage Solr interactions.
• No built-in delta sync: You have to implement incremental logic yourself (e.g., tracking last sync timestamps or listening for database changes).

• Go with DIH if: You need scheduled batch syncs, have simple data mapping requirements, and don't need complex business logic during indexing. It's the fastest way to get up and running.
• Go with SolrNet if: You need real-time indexing, have complex data transformations/aggregation across the two databases, or want to tie indexing directly to your app's business workflows. It offers maximum flexibility.

Example Implementations #

DIH Configuration #

1. Enable DIH in solrconfig.xml:

<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
  <lst name="defaults">
    <str name="config">data-config.xml</str>
  </lst>
</requestHandler>

2. Create data-config.xml for two databases:

<dataConfig>
  <!-- Data source for first database -->
  <dataSource name="db1" type="JdbcDataSource" 
              driver="com.microsoft.sqlserver.jdbc.SQLServerDriver"
              url="jdbc:sqlserver://your-db1-server;databaseName=EventDB1"
              user="dbuser1" password="dbpass1"/>
  
  <!-- Data source for second database -->
  <dataSource name="db2" type="JdbcDataSource" 
              driver="com.microsoft.sqlserver.jdbc.SQLServerDriver"
              url="jdbc:sqlserver://your-db2-server;databaseName=EventDB2"
              user="dbuser2" password="dbpass2"/>

  <document>
    <!-- Import events from db1 -->
    <entity name="event_db1" dataSource="db1" 
            query="SELECT Id, Title, EventDate, Description FROM Events">
      <field column="Id" name="event_id"/>
      <field column="Title" name="event_title"/>
      <field column="EventDate" name="event_date"/>
      <field column="Description" name="event_description"/>
      <field name="source_db" value="db1"/> <!-- Tag source database -->
    </entity>

    <!-- Import events from db2 -->
    <entity name="event_db2" dataSource="db2" 
            query="SELECT EventId, EventName, OccurDate, Details FROM EventRecords">
      <field column="EventId" name="event_id"/>
      <field column="EventName" name="event_title"/>
      <field column="OccurDate" name="event_date"/>
      <field column="Details" name="event_description"/>
      <field name="source_db" value="db2"/> <!-- Tag source database -->
    </entity>
  </document>
</dataConfig>

Run a full import by visiting:
http://your-solr-host:8983/solr/your-core/dataimport?command=full-import

SolrNet Implementation in ASP.NET MVC 5 #

1. Install SolrNet via NuGet:

Install-Package SolrNet

2. Initialize SolrNet in Global.asax.cs:

protected void Application_Start()
{
    AreaRegistration.RegisterAllAreas();
    FilterConfig.RegisterGlobalFilters(GlobalFilters.Filters);
    RouteConfig.RegisterRoutes(RouteTable.Routes);
    BundleConfig.RegisterBundles(BundleTable.Bundles);

    // Initialize Solr connection
    Startup.Init<EventSolrDocument>("http://your-solr-host:8983/solr/your-core");
}

3. Define Solr Document Class:

public class EventSolrDocument
{
    [SolrUniqueKey("event_id")]
    public string Id { get; set; }

    [SolrField("event_title")]
    public string Title { get; set; }

    [SolrField("event_date")]
    public DateTime EventDate { get; set; }

    [SolrField("event_description")]
    public string Description { get; set; }

    [SolrField("source_db")]
    public string SourceDb { get; set; }
}

4. Indexing Service Class:

public class EventIndexingService
{
    private readonly ISolrOperations<EventSolrDocument> _solr;
    private readonly EventDb1Context _db1Context; // Your EF context for db1
    private readonly EventDb2Context _db2Context; // Your EF context for db2

    public EventIndexingService(ISolrOperations<EventSolrDocument> solr, 
                                EventDb1Context db1Context, 
                                EventDb2Context db2Context)
    {
        _solr = solr;
        _db1Context = db1Context;
        _db2Context = db2Context;
    }

    // Full index of all events from both databases
    public void FullIndexAllEvents()
    {
        var db1Events = _db1Context.Events.Select(e => new EventSolrDocument
        {
            Id = e.Id.ToString(),
            Title = e.Title,
            EventDate = e.EventDate,
            Description = e.Description,
            SourceDb = "db1"
        }).ToList();

        var db2Events = _db2Context.EventRecords.Select(er => new EventSolrDocument
        {
            Id = er.EventId.ToString(),
            Title = er.EventName,
            EventDate = er.OccurDate,
            Description = er.Details,
            SourceDb = "db2"
        }).ToList();

        // Clear existing index (optional) and push new data
        _solr.Delete(SolrQuery.All);
        _solr.AddRange(db1Events.Concat(db2Events));
        _solr.Commit();
    }

    // Incremental index: sync only updated events since last index
    public void IncrementalIndexEvents(DateTime lastSyncTime)
    {
        var db1UpdatedEvents = _db1Context.Events
            .Where(e => e.UpdatedAt > lastSyncTime)
            .Select(e => new EventSolrDocument
            {
                Id = e.Id.ToString(),
                Title = e.Title,
                EventDate = e.EventDate,
                Description = e.Description,
                SourceDb = "db1"
            }).ToList();

        var db2UpdatedEvents = _db2Context.EventRecords
            .Where(er => er.LastModified > lastSyncTime)
            .Select(er => new EventSolrDocument
            {
                Id = er.EventId.ToString(),
                Title = er.EventName,
                EventDate = er.OccurDate,
                Description = er.Details,
                SourceDb = "db2"
            }).ToList();

        var updatedEvents = db1UpdatedEvents.Concat(db2UpdatedEvents);
        if (updatedEvents.Any())
        {
            _solr.AddRange(updatedEvents);
            _solr.Commit();
        }
    }
}

5. Use the Service in a Controller:

public class EventAdminController : Controller
{
    private readonly EventIndexingService _indexingService;

    public EventAdminController(EventIndexingService indexingService)
    {
        _indexingService = indexingService;
    }

    public ActionResult TriggerFullIndex()
    {
        _indexingService.FullIndexAllEvents();
        return Content("Full index completed successfully!");
    }

    public ActionResult TriggerIncrementalIndex()
    {
        // Fetch last sync time from config/database (example value here)
        var lastSync = DateTime.Now.AddHours(-24);
        _indexingService.IncrementalIndexEvents(lastSync);
        return Content("Incremental index completed successfully!");
    }
}

内容的提问来源于stack exchange，提问作者Gaurav Juyal