约8%日期时间字符串转换年月错误,求技术排查解决
日期转换错误排查求助
我需要计算起止日期之间的时长,但日期格式不固定。我编写了字符串清理和多格式解析函数,但约8%的日期转换不准确,出现119条结束日期早于开始日期的错误记录,无法定位问题。
测试数据包含多种格式的起止日期,错误转换的记录也已整理出来,以下是我的代码:
data <- read.csv('justDates.csv', na.strings=c("","NA")) ## FUNCTIONS ----------------------------------------- # clean text # don't judge me, I hate regex and this is hella ugly but gets the job done. Maybe. clean_dates <- function(datetime_string) { # Normalize timezones by removing all mentions of timezone abbreviations except UTC datetime_string2 <- gsub("\\s*\\([^\\)]+\\)", "", datetime_string) # Remove extra timezone notes in parentheses datetime_string3 <- gsub("(?<=\\d)(?=UTC|\\sUTC).*", "", datetime_string2, perl = TRUE) # Strip trailing text after UTC datetime_string4 <- gsub("[\n\t]", " ", datetime_string3, perl = TRUE) # Remove newlines and tabs datetime_string5 <- gsub("UTC\\s+", "UTC", datetime_string4, perl = TRUE) # Clean trailing whitespace after UTC datetime_string6 <- gsub(" ?(UTC|GMT|UT)", "", datetime_string5, perl = TRUE) datetime_string7 <- gsub("[\\h\\v]+$","",datetime_string6, perl=TRUE) return(datetime_string7) } #convert various date and time formats into one standard format parse_date <- function(date_string){ parse_attempts <- list( function(x) mdy_hms(x), function(x) dmy_hms(x), function(x) mdy_hm(x), function(x) dmy_hm(x), function(x) ymd_hms(x), function(x) ymd_hm(x) ) # # Attempt each format, catching errors and returning the first successful conversion for (parse_fn in parse_attempts) { date <- tryCatch(parse_fn(date_string), error = function(e) NA) if (!is.na(date)) return(as.numeric(as.POSIXct(date))) # Convert to consistent POSIXct format if successful } return(NA) # Return NA if no formats matched # Attempt to parse the date with each format } # Clean the Dates df_dates <- data |> mutate( start_date_clean=sapply(start_date,clean_dates), end_date_clean=sapply(end_date,clean_dates) ) # Convert to POSIXct df_dates_posix <- df_dates |> mutate( start_datePOSIX = sapply(start_date_clean, parse_date), end_datePOSIX= sapply(end_date_clean, parse_date) ) # Check dates are correct df_dates_comp <- df_dates_posix |> mutate( neg=end_datePOSIX<=start_datePOSIX, start_date_rev=as_datetime(start_datePOSIX), end_date_rev=as_datetime(end_datePOSIX) ) test3 |> filter(neg==TRUE) |> nrow()
经检查有119条记录的end_date早于start_date,对比*_date_rev与*_date_clean列可发现转换错误,但无法定位原因。
恳请各位提供帮助与见解,我哪里出错了?
内容的提问来源于stack exchange,提问作者zaneywolf
相关产品推荐
相关产品推荐

