多第三方数据源接入场景下REST API数据同步应用的架构模式选型咨询

Great question—your team's pain point with Onion Architecture here makes total sense. When your primary workflow is shuttling data between external REST APIs, forcing every transformation through a generic core layer can feel like unnecessary red tape, especially when those intermediate core objects don't add meaningful business value. Let's break down some alternative architectures that fit your use case perfectly:

1. Pipeline Architecture（管道架构） #

This pattern is tailor-made for data-centric workflows like syncing, as it organizes logic into sequential, reusable steps:

• Split your sync process into discrete, focused components:
  • Source Fetch: Pull raw data from the third-party API (lives in the infrastructure/adapter layer)
  • Delta Check: Compare source data with existing target records to identify new/updated entries (keep this in the core layer if it's a universal sync rule, or in infrastructure if it's API-specific)
  • Data Transformation: Convert source models to target models—you can skip the core layer middleman entirely here if there's no shared business logic to apply, mapping directly between external objects
  • Target Push: Send transformed data to the destination API (infrastructure/adapter layer)
• The core layer only needs to define the pipeline contract (e.g., an ISyncStep interface, a SyncContext object to pass state between steps) and universal rules like retry policies or error handling.
• Pros: Extremely flexible for adding new sources/targets (just plug in new pipeline steps), eliminates redundant core object mapping when it doesn't add value.

2. Hexagonal Architecture（六边形/端口与适配器架构） #

While it shares some principles with Onion Architecture, Hexagonal is far more forgiving about how external systems interact with your core logic:

• Define ports in the core layer that represent actions, not data structures. For example:
  • IDataSource with a method GetRecentRecords(DateTime lastSyncTimestamp)
  • IDataTarget with a method UpsertRecords(IEnumerable<object> records)
• Build adapters in the infrastructure layer that implement these ports for each third-party API. An adapter can handle:
  • Calling the external API's endpoints
  • Converting the external API's model to whatever format the core needs (or even directly to the target API's model, if the core doesn't need to inspect the data)
• The core layer focuses solely on the sync logic itself: "fetch recent records from the source, upsert them to the target"—it doesn't care about the specific data shapes, just that adapters fulfill the port contracts.
• Pros: Keeps the core layer lean (no redundant data models) while maintaining full decoupling from external APIs. Adding new sources/targets is as simple as writing a new adapter.

3. Event-Driven Architecture with Mediator Pattern #

If you're planning to scale to multiple sources and targets long-term, this pattern will make your system highly extensible and decoupled:

• The core layer defines events (e.g., SourceDataFetchedEvent, RecordsReadyForUpsertEvent) and a mediator that routes events to the appropriate handlers.
• Infrastructure layer components:
  • Source Adapters: Fetch data from third-party APIs, then publish a SourceDataFetchedEvent with the raw source data.
  • Transformation Handlers: Listen for SourceDataFetchedEvent, convert source data to target data (skip core objects if no shared logic is needed), then publish RecordsReadyForUpsertEvent.
  • Target Adapters: Listen for RecordsReadyForUpsertEvent and push data to the destination API.
• You can add universal handlers for cross-cutting concerns like logging or error tracking, or multiple transformation handlers for different source-target pairs.
• Pros: Fully decoupled components—adding a new source just requires a new event publisher, adding a new target requires a new event subscriber. No forced core object mapping unless you need shared validation or business rules.

Practical Quick Wins #

• Skip the middleman when possible: If your core layer doesn't need to process the data (no validation, no business rules), don't waste time mapping to core objects. Let adapters handle direct source-to-target conversion.
• Extract universal logic to core: Keep the core layer focused on what's truly shared across all sync workflows—like delta calculation logic, retry policies, or compliance checks. Leave API-specific logic in the infrastructure layer.
• Use mapping tools: If you do need to map between models (even core ones), tools like AutoMapper can reduce boilerplate and make the overhead worth it.

内容的提问来源于stack exchange，提问作者Hyperwave