Julia 0.6.2中RemoteRef未定义问题求助(原0.4.7可运行)
解决Julia 0.6.2中"RemoteRef未定义"的并行代码适配问题
从Julia 0.4.7升级到0.6.2后,并行编程API有不少关键变化,你遇到的RemoteRef未定义错误就是因为这些改动导致的。下面我给你梳理具体的解决思路和修改后的代码:
核心原因与关键改动说明
- Julia 0.5+版本开始,并行相关的类型(比如
RemoteRef)和宏(比如@spawn)都被迁移到了Distributed标准库中,必须显式导入才能使用,不再是全局默认可用的。 @spawn在0.6中返回的是Future类型(属于RemoteRef的子类),所以原来用Array(RemoteRef, nprocs)定义结果数组的旧式语法已经失效,需要改用Vector{Future}。- 所有
@everywhere修饰的函数,必须确保所有工作进程都能加载到依赖的模块(比如RasterIO、DataFrames),否则会出现找不到函数/类型的错误。
具体修复步骤
- 导入Distributed模块并启动工作进程:在代码开头添加
using Distributed,同时用addprocs()启动对应数量的工作进程(和你代码中预设的进程数匹配)。 - 统一依赖加载逻辑:把包的安装和模块导入放在
@everywhere块中,确保所有工作进程都能获取到依赖。 - 替换
RemoteRef数组类型:将references = Array(RemoteRef, nprocs)改为references = Vector{Future}(undef, nprocs),适配0.6的类型系统。 - 移除手动定义
RemoteRef的尝试:这在0.6中完全没必要,反而会导致类型不匹配的错误。
修改后的完整代码
# 导入并行模块并启动3个工作进程 using Distributed addprocs(3) # 在所有工作进程上安装并加载依赖包 @everywhere begin Pkg.add("DataFrames") Pkg.clone("git://github.com/wkearn/RasterIO.jl") Pkg.build("RasterIO") using DataFrames using RasterIO end output_file = "point_irradiance.asc" dtm_file = "///dtm.tif" dsm_file = "///dsm.tif" buildings_file = "///build.asc" extent_file = "///extent.dat" lamps_file = "////pen_light.csv" lamp_easting = 3 lamp_northing = 4 lamp_height = 2 @everywhere function lightdist(xdist,ydist,zdist) xyzdist = sqrt(xdist^2+ydist^2+zdist^2) return (1/(xyzdist^2)) end function create_point_irradiance() # Read in locations of street maps lamps = convert(Matrix, readtable(lamps_file)) extent = convert(Matrix, DataFrames.readtable(extent_file)) xindex = collect(extent[1,1]:1:(extent[1,2]-1)) yindex = collect(extent[1,4]:-1:(extent[1,3]+1)) xindexmax, = size(xindex) yindexmax, = size(yindex) nlamps, = size(lamps) nprocs = nworkers() # 改用nworkers()获取实际工作进程数,更灵活 chunk = div(nlamps, nprocs) # 替换为Vector{Future}类型存储@spawn的返回结果 references = Vector{Future}(undef, nprocs) for core in 1:nprocs from = core == 1 ? 1 : (chunk * (core - 1)) + 1 to = core == nprocs ? nlamps : chunk * core # @spawn在Distributed模块导入后可直接使用 references[core] = @spawn calculate_shadow(lamps[from:to, :], dtm_file, dsm_file, buildings_file, xindex, yindex, xindexmax, yindexmax, lamp_easting, lamp_northing, lamp_height) end point_irradiance = fill(0.0, (yindexmax, xindexmax)) for core in 1:nprocs point_irradiance += fetch(references[core]) end writedlm(output_file, point_irradiance, " ") function add_headers(f::IOStream) return string("NCOLS ", xindexmax, "\n", "NROWS ", yindexmax, "\n", "XLLCORNER ", extent[1,1], "\n", "YLLCORNER ", extent[1,3], "\n", "CELLSIZE 2 \n", "NODATA_VALUE -9999\n", readall(f)) end content = open(add_headers, output_file); f = open(output_file, "w") write(f, content) close(f) end @everywhere function calculate_shadow(lamps, dtm_file, dsm_file, buildings_file, xindex, yindex, xindexmax, yindexmax, lamp_easting, lamp_northing, lamp_height) ext = 100 lamp_ext = 100 absorbance = 0.1 sensor_ht = 2.5 dtmR = RasterIO.openraster(dtm_file, UInt32(0)) dtm = transpose(RasterIO.fetch(dtmR, 1)) RasterIO.closeraster(dtmR) dsmR = RasterIO.openraster(dsm_file, UInt32(0)) dsm = transpose(RasterIO.fetch(dsmR, 1)) RasterIO.closeraster(dsmR) buildingsR = RasterIO.openraster(buildings_file, convert(UInt32, 0)) buildings = transpose(RasterIO.fetch(buildingsR, 1)) RasterIO.closeraster(buildingsR) # Produce raster of structures (buildings, trees, hedges, etc) surf = dsm - dtm buildings = [if (buildings[ydim, xdim] == 0) 0 else surf[ydim, xdim] end for ydim =1:yindexmax, xdim=1:xindexmax] hard_surf = dtm + buildings soft_surf = surf - buildings # release memory dsm = 0 surf = 0 buildings = 0 point_irradiance = fill(0.0, size(hard_surf)) for lamp in 1:size(lamps, 1) x = lamps[lamp,lamp_easting] y = lamps[lamp,lamp_northing] z = lamps[lamp,lamp_height] Xmax = xindex[xindexmax, 1] Xmin = xindex[1, 1] Ymin = yindex[yindexmax, 1] Ymax = yindex[1, 1] if ((x <= (Xmax+ext)) & (x>= (Xmin-ext)) & (y<=(Ymax+ext)) & (y>=(Ymin-ext))) row = find(yindex .== min(max(y,Ymin),Ymax))[1] col = find(xindex .== max(min(x,Xmax),Xmin))[1] col_min = max(col-ext,1) col_max = min(col+ext, xindexmax) col_lamp = col-col_min row_min = max(row-ext, 1) row_max = min(row+ext, yindexmax) row_lamp = row-row_min nrows = row_max-row_min ncols = col_max-col_min hard_block = hard_surf[row_min:(row_min + nrows - 1),col_min:(col_min + ncols - 1)] soft_block = soft_surf[row_min:(row_min + nrows - 1),col_min:(col_min + ncols - 1)] terrain_block = dtm[row_min:(row_min + nrows - 1),col_min:(col_min + ncols - 1)] point_irrad = fill(0.0, size(hard_block)) if (ncols==200&nrows==200) for xx in 1:ncols for yy in 1:nrows xdist = col_lamp-xx ydist = row_lamp-yy xydist = sqrt(xdist^2+ydist^2) zdist = (terrain_block[row_lamp,col_lamp]+z)-(terrain_block[yy,xx]+sensor_ht) xyzdist = sqrt(xydist^2+zdist^2) dist = floor(xydist+0.5) if (xydist<=lamp_ext) if (zdist>0) shadow = 1 shading = 0 if(dist>0) for d in 1:dist if (hard_block[round(Int, yy+(ydist)*(d/dist)),round(Int, xx+(xdist)*(d/dist))] >= (terrain_block[yy,xx]+sensor_ht+(d/dist)*zdist)) shadow = 0 end if(soft_block[round(Int, yy+(ydist)*(d/dist)),round(Int, (xx+(xdist)*(d/dist)))] >= (terrain_block[yy,xx]+sensor_ht+(d/dist)*zdist)) shading = shading + xyzdist/xydist end end end res = (1/(10^(absorbance*shading)))*shadow*lightdist(xdist,ydist,zdist) end end end end end end end return point_irradiance end
额外优化提示
- 我把
nprocs=3改成了nworkers(),这样代码会自动匹配你启动的工作进程数,避免硬编码带来的不一致。 - Julia 0.6中
readall(f)已经被标记为弃用,建议替换为read(f),如果需要兼容现有逻辑可以暂时保留。 - 如果运行时还出现模块找不到的错误,可以检查
RasterIO包在0.6版本下的兼容性,必要时可以寻找替代包或者调整包的版本。
内容的提问来源于stack exchange,提问作者DFinch
相关产品推荐
相关产品推荐

