PHP中如何在不使峰值内存翻倍的情况下转换大数组?
我正在寻找内存效率最高的方法,处理已在内存中的大型PHP数组,将其转换为另一种格式,同时尽可能减少峰值内存使用量的增长,且不将数据写入磁盘。
最初我认为在遍历$source_array时,只需unset()每一行即可释放内存,但实际上这反而增加了峰值内存使用量。以下是展示该情况的示例代码:
$source_array = array_fill( 0, 500000, [ 'col1' => 'val1', 'col2' => 'val2', 'col3' => 'val3', 'col4' => 'val4', 'col5' => 'val5',] ); echo "Initial Memory Usage: ". memory_get_usage() .' Peak Memory Usage: '. memory_get_peak_usage() ."<br>\n"; $destination_array = []; foreach( $source_array as $key => $row ) { $destination_array[$key] = $row; //Assume some manipulation of $row here. //$source_array[$key] = null; //Causes peak memory to increase. //unset( $source_array[$key] ); //Causes peak memory to increase if ( $key > 0 && $key % 100000 == 0 ) { echo " $key. Inside Loop Memory Usage: ". memory_get_usage() .' Inside Loop Peak Memory Usage: '. memory_get_peak_usage() ."<br>\n"; //Peak memory usage is double initial memory usage. gc_collect_cycles(); } } unset( $source_array ); echo "Final Memory Usage: ". memory_get_usage() .' Final Peak Memory Usage: '. memory_get_peak_usage() ."<br>\n"; //Peak memory usage is double initial memory usage.
输出注意:初始和最终内存使用量相同,但峰值内存翻倍。
Initial Memory Usage: 17250856 Peak Memory Usage: 17250952
100000. Inside Loop Memory Usage: 21449400 Inside Loop Peak Memory Usage: 21449520
200000. Inside Loop Memory Usage: 25643704 Inside Loop Peak Memory Usage: 25643824
300000. Inside Loop Memory Usage: 34032312 Inside Loop Peak Memory Usage: 34032432
400000. Inside Loop Memory Usage: 34032312 Inside Loop Peak Memory Usage: 34032512
Final Memory Usage: 17250856 Final Peak Memory Usage: 34032512
目前我想到的最佳方法是使用SplFixedArray,代码如下:
$source_array = array_fill( 0, 500000, [ 'col1' => 'val1', 'col2' => 'val2', 'col3' => 'val3', 'col4' => 'val4', 'col5' => 'val5',] ); echo "Initial Memory Usage: ". memory_get_usage() .' Peak Memory Usage: '. memory_get_peak_usage() ."<br>\n"; $source_array = SplFixedArray::fromArray( $source_array ); echo "Pre-Loop Memory Usage: ". memory_get_usage() .' Pre-Loop Peak Memory Usage: '. memory_get_peak_usage() ."<br>\n"; $destination_array = new SplFixedArray( count( $source_array ) ); //This reduces peak memory by about 20-30%. foreach( $source_array as $key => $row ) { $destination_array[$key] = $row; //Assume some manipulation of $row here. //$source_array[$key] = null; //Causes peak memory to increase. //unset( $source_array[$key] ); //Causes peak memory to increase if ( $key > 0 && $key % 100000 == 0 ) { echo " $key. Inside Loop Memory Usage: ". memory_get_usage() .' Inside Loop Peak Memory Usage: '. memory_get_peak_usage() ."<br>\n"; //Peak memory usage is double initial memory usage. gc_collect_cycles(); } } unset( $source_array ); echo "Final Memory Usage: ". memory_get_usage() .' Final Peak Memory Usage: '. memory_get_peak_usage() ."<br>\n"; //Peak memory usage is double initial memory usage.
输出:
Initial Memory Usage: 17250920 Peak Memory Usage: 17251016
Pre-Loop Memory Usage: 8473216 Pre-Loop Peak Memory Usage: 25254608
100000. Inside Loop Memory Usage: 16477064 Inside Loop Peak Memory Usage: 25254608
200000. Inside Loop Memory Usage: 16477120 Inside Loop Peak Memory Usage: 25254608
300000. Inside Loop Memory Usage: 16477120 Inside Loop Peak Memory Usage: 25254608
400000. Inside Loop Memory Usage: 16477120 Inside Loop Peak Memory Usage: 25254608
Final Memory Usage: 8473272 Final Peak Memory Usage: 25254608
虽然相比原始方法有约30%的改进,但我认为应该可以实现接近零的峰值内存增长,请问有什么思路吗?
内容的提问来源于stack exchange,提问作者Mike

