IIS部署Hangfire服务运行一段时间后失败排查求助

已搭建包含服务器和仪表盘的Hangfire服务并部署至IIS，创建了名为HangfireServerDb的数据库。启动IIS后，Hangfire服务器成功生成所需架构和表，初期所有任务运行正常。但运行一段时间后，仪表盘显示调度任务出现如下错误：

System.InvalidOperationException: 无法调度循环作业，详情请查看内部异常。
---> Hangfire.Common.JobLoadException: 无法加载作业，详情请查看内部异常。
---> System.IO.FileNotFoundException: 无法解析程序集 'HangfireDashboard.WebApi'。

at System.TypeNameParser.ResolveAssembly(String asmName, Func`2 assemblyResolver, Boolean throwOnError, StackCrawlMark& stackMark)
   at System.TypeNameParser.ConstructType(Func`2 assemblyResolver, Func`4 typeResolver, Boolean throwOnError, Boolean ignoreCase, StackCrawlMark& stackMark)
   at System.TypeNameParser.GetType(String typeName, Func`2 assemblyResolver, Func`4 typeResolver, Boolean throwOnError, Boolean ignoreCase, StackCrawlMark& stackMark)
   at System.Type.GetType(String typeName, Func`2 assemblyResolver, Func`4 typeResolver, Boolean throwOnError)
   at Hangfire.Common.TypeHelper.DefaultTypeResolver(String typeName)
   at Hangfire.Storage.InvocationData.DeserializeJob()
   --- 内部异常堆栈跟踪结束 ---
   at Hangfire.Storage.InvocationData.DeserializeJob()
   at Hangfire.RecurringJobEntity..ctor(String recurringJobId, IDictionary`2 recurringJob, ITimeZoneResolver timeZoneResolver, DateTime now)
   --- 内部异常堆栈跟踪结束 ---
   at Hangfire.Server.RecurringJobScheduler.ScheduleRecurringJob(BackgroundProcessContext context, IStorageConnection connection, String recurringJobId, RecurringJobEntity recurringJob, DateTime now).

相关代码及排查信息如下：

HttpClient注册代码 #

services.AddHttpClient<WebPlatformHttpClient>(
                httpClient =>
                {
                    httpClient.BaseAddress = new Uri(_configuration["WebPlatform:Api:BaseUri"]);
                    httpClient.DefaultRequestHeaders.Clear();
                    httpClient.DefaultRequestHeaders.Add(HeaderNames.Accept, "application/json");
                });

HttpClient实现代码 #

public class WebPlatformHttpClient
{
    private readonly HttpClient _httpClient;
    private readonly IConfiguration _configuration;

    public WebPlatformHttpClient(HttpClient httpClient, IConfiguration configuration)
    {
        _httpClient = httpClient;
        _configuration = configuration;
    }

    public async Task SendEmailAsync()
    {
        HttpResponseMessage response = await _httpClient.GetAsync(_configuration["WebPlatform:Api:SendEmail"]);

        response.EnsureSuccessStatusCode();
    }
}

定时任务代码 #

RecurringJob.AddOrUpdate(
            $"Send Email Job",
            "web-platform",
            (WebPlatformHttpClient httpClient) => httpClient.SendEmailAsync(),
            configuration["WebPlatform:Crons:SendEmailCron"], 
            jobOptions);

后续排查到的进程终止异常信息：

Application: w3wp.exe
CoreCLR Version: 6.0.2623.60508
.NET Version: 6.0.26
Description: 进程因未处理的异常而终止。
Exception Info: System.AggregateException: 一个或多个托管服务停止失败。(任务已取消。)
---> System.Threading.Tasks.TaskCanceledException: 任务已取消。
at Hangfire.Processing.TaskExtensions.WaitOneAsync(WaitHandle waitHandle, TimeSpan timeout, CancellationToken token) in C:\projects\hangfire-525\src\Hangfire.Core\Processing\TaskExtensions.cs:line 96
at Hangfire.Processing.BackgroundDispatcher.WaitAsync(TimeSpan timeout, CancellationToken cancellationToken) in C:\projects\hangfire-525\src\Hangfire.Core\Processing\BackgroundDispatcher.cs:line 82
at Hangfire.Server.BackgroundProcessingServer.WaitForShutdownAsync(CancellationToken cancellationToken) in C:\projects\hangfire-525\src\Hangfire.Core\Server\BackgroundProcessingServer.cs:line 148
at Microsoft.Extensions.Hosting.Internal.Host.StopAsync(CancellationToken cancellationToken)
--- 内部异常堆栈跟踪结束 ---
at Microsoft.Extensions.Hosting.Internal.Host.StopAsync(CancellationToken cancellationToken)
at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.WaitForShutdownAsync(IHost host, CancellationToken token)
at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
at HangfireDashboard.WebApi.Program.Main(String[] args) in D:\CI_Data\TeamCity\buildAgent2\work\ea4fb37391903d0f\Services\HangfireDashboard\Program.cs:line 22
at HangfireDashboard.WebApi.Program.

(String[] args)

核心原因 #

1. 程序集解析失败：Hangfire存储的循环作业元数据中记录了程序集名称HangfireDashboard.WebApi，但后续运行时IIS进程无法找到该程序集。常见触发场景包括：
   • IIS应用池回收后，程序集加载路径变化或文件丢失
   • 部署时程序集未正确复制到输出目录
   • 应用程序域重启后，默认类型解析器无法定位程序集
2. 进程终止连锁反应：后续出现的TaskCanceledException是Hangfire服务器在IIS应用池回收时的正常终止流程被中断导致的，属于程序集加载失败引发的次生问题。

解决方案 #

1. 确保程序集正确部署 #

• 检查IIS站点bin目录下是否存在HangfireDashboard.WebApi.dll及对应.pdb文件
• 确认发布目标框架与运行环境一致（当前为.NET 6），避免发布时遗漏依赖
• 在项目属性的“生成”选项中，勾选“复制本地”确保所有依赖程序集被复制到输出目录

2. 自定义Hangfire类型解析器 #

默认类型解析器依赖System.Type.GetType，可能存在加载路径问题，可自定义解析器指定程序集加载逻辑：

public class CustomTypeResolver : ITypeResolver
{
    public Type ResolveType(string typeName)
    {
        // 优先从当前应用程序域加载目标程序集
        var assembly = Assembly.Load("HangfireDashboard.WebApi");
        return assembly.GetType(typeName) ?? Type.GetType(typeName);
    }
}

// 在Hangfire配置中注册自定义解析器
services.AddHangfire(config =>
{
    config.UseSqlServerStorage("HangfireServerDb");
    config.UseTypeResolver(new CustomTypeResolver());
});

3. 修复循环作业注册方式 #

当前注册方式的委托序列化依赖程序集信息，建议改用强类型注册减少依赖：

RecurringJob.AddOrUpdate<WebPlatformHttpClient>(
    "Send Email Job",
    client => client.SendEmailAsync(),
    configuration["WebPlatform:Crons:SendEmailCron"],
    queue: "web-platform",
    options: jobOptions);

4. 优化IIS应用池配置 #

• 调整应用池回收策略，避免频繁回收（如设置固定时间间隔而非内存限制）
• 启用“加载用户配置文件”选项，确保程序集加载权限正确
• 设置更长的闲置超时时间，减少不必要的进程终止

内容的提问来源于stack exchange，提问作者user23313952