使用Skrapeit自定义Fetcher时Ktor request函数参数不匹配报错

问题分析 #

按照skrape{it}自定义Fetcher文档实现代码时，出现HttpClient.request方法参数不匹配的编译错误，尽管IDE识别到了对应的重载方法。核心原因是Ktor版本与文档示例版本不兼容，新版本Ktor对request方法的签名或调用逻辑做了调整；同时代码中还存在Cookie类型转换的潜在问题。

修复方案 #

1. 调整Ktor请求调用方式 #

将直接传入HttpRequestBuilder的写法，改为通过lambda导入已有请求配置的方式，适配新版本Ktor的API：

override fun fetch(request: HttpRequestBuilder): Result =
    runBlocking {
        // 改用lambda方式导入已构建的request配置
        with(ktorClient.request<HttpResponse> { takeFrom(request) }) {
            Result(
                responseBody = readText(),
                responseStatus = Result.Status(status.value, status.description),
                contentType = contentType()?.toString(),
                headers = headers.toMap().mapValues { it.value.firstOrNull().orEmpty() },
                baseUri = request.url.toString(),
                // 修复Cookie类型转换问题：映射Ktor Cookie到skrape的Cookie
                cookies = headers.values(HttpHeaders.SetCookie).mapNotNull { cookieHeader ->
                    io.ktor.http.Cookie.parse(cookieHeader).firstOrNull()?.let { ktorCookie ->
                        Cookie(
                            name = ktorCookie.name,
                            value = ktorCookie.value,
                            domain = ktorCookie.domain,
                            path = ktorCookie.path,
                            expires = ktorCookie.expires,
                            maxAge = ktorCookie.maxAge,
                            secure = ktorCookie.secure,
                            httpOnly = ktorCookie.httpOnly
                        )
                    }
                }
            )
        }
    }

2. 确认依赖版本一致性 #

确保项目中Ktor版本与skrape{it}兼容，比如使用skrape{it}最新版（如1.2.0+）时，搭配Ktor 2.x版本，在build.gradle.kts或build.gradle中统一版本：

// build.gradle.kts示例
val ktorVersion = "2.3.3"
dependencies {
    implementation("io.ktor:ktor-client-core:$ktorVersion")
    implementation("io.ktor:ktor-client-apache:$ktorVersion")
    implementation("it.skrape:skrapeit-core:1.2.0")
}

3. 清理导入与缓存 #

• 检查是否有重复或错误的导入，确保HttpRequestBuilder来自io.ktor.client.request包
• 执行IDE清理缓存操作（如IntelliJ的File -> Invalidate Caches...）后重新编译项目

完整修复后的代码 #

import it.skrape.fetcher.*
import io.ktor.client.request.*
import io.ktor.client.statement.*
import io.ktor.client.*
import io.ktor.client.engine.apache.*
import io.ktor.http.*
import kotlinx.coroutines.runBlocking

class KtorBlockingFetcher(val ktorClient: HttpClient) : BlockingFetcher<HttpRequestBuilder> {

    override fun fetch(request: HttpRequestBuilder): Result =
        runBlocking {
            with(ktorClient.request<HttpResponse> { takeFrom(request) }) {
                Result(
                    responseBody = readText(),
                    responseStatus = Result.Status(status.value, status.description),
                    contentType = contentType()?.toString(),
                    headers = headers.toMap().mapValues { it.value.firstOrNull().orEmpty() },
                    baseUri = request.url.toString(),
                    cookies = headers.values(HttpHeaders.SetCookie).mapNotNull { cookieHeader ->
                        io.ktor.http.Cookie.parse(cookieHeader).firstOrNull()?.let { ktorCookie ->
                            Cookie(
                                name = ktorCookie.name,
                                value = ktorCookie.value,
                                domain = ktorCookie.domain,
                                path = ktorCookie.path,
                                expires = ktorCookie.expires,
                                maxAge = ktorCookie.maxAge,
                                secure = ktorCookie.secure,
                                httpOnly = ktorCookie.httpOnly
                            )
                        }
                    }
                )
            }
        }

    override val requestBuilder: HttpRequestBuilder
        get() = HttpRequestBuilder()
}

val KTOR_CLIENT = HttpClient(Apache)
val result = skrape(KtorBlockingFetcher(KTOR_CLIENT)) {
    request {
        url("https://google.com") // 简化URL写法，无需嵌套Url构造器
    }

    response {
        // 处理响应逻辑，比如打印响应体
        println(responseBody)
    }
}

内容的提问来源于stack exchange，提问作者DannyBunny