Django视图中递归函数无返回结果问题排查求助

问题描述 #

我是Django和Web开发新手，点击表单提交按钮时尝试在views.py中调用递归函数。当函数返回简单列表时运行正常，但启用核心逻辑后无任何返回结果，不清楚问题所在，请求帮忙排查问题并给出解决方法。

相关代码如下：

views.py实现 #

def process_form(request):
    if request.method == 'POST':
        print("Request is POST")
        form = UIForm(request.POST, request.FILES)
        if form.is_valid():
            print("Form is valid")
            # Create an instance of the model and assign the form data to its fields
            form_data = FormData()
            form_data.from_timestamp = form.cleaned_data['from_timestamp']
            form_data.to_timestamp = form.cleaned_data['to_timestamp']
            form_data.free_text = form.cleaned_data['free_text']
            form_data.uploaded_file = form.cleaned_data['file']

            # Save the form data to the database
            form_data.save()
            file = request.FILES['file']

            todayDate = datetime.datetime.now().strftime("%Y-%m-%d")
            dest_dir = temp_directory + os.sep + todayDate
            dest_path = dest_dir + os.sep + file.name
            os.makedirs(dest_dir, exist_ok=True)
            print("Dest path:", dest_path)
            # Handle the uploaded file
            if file:
                # Save the file to the server's temporary directory
                with open(f'{dest_path}', 'wb') as destination:
                    for chunk in file.chunks():
                        destination.write(chunk)

            # Check the given regular expression against the extracted files
            pattern = form_data.free_text
            extracted_dir = extract_files_and_return_list_of_paths(dest_path)
            matching_files = search_files_for_pattern(extracted_dir, pattern)

            print("matching_files:", matching_files)
            for i in range(len(matching_files)):
                print("matching_files[", i, "]:", matching_files[i])

            # Store the matching_files list in the session
            request.session['matching_files'] = matching_files

            # Create the JSON response data
            response_data = {
                'success': True,
                'pk': form_data.pk,
                'matching_files': matching_files
            }
            return JsonResponse(response_data)
        else:
            # Return JSON response with errors if the form is invalid
            errors = json.loads(form.errors.as_json())
            return JsonResponse({'success': False, 'errors': errors})

    else:
        form = UIForm()

    return render(request, 'ui.html', {'form': form})

递归函数（原问题代码） #

def search_files_for_pattern(directory, pattern):
    matching_files = list()
    for filename in os.listdir(directory):
        file_path = os.path.join(directory, filename)
        if os.path.isfile(file_path):  # Filter for text files
            with open(file_path, 'r', encoding='utf-8', errors='ignore') as file:
                file_contents = file.read()
                if re.search(pattern, file_contents):
                    #print("Found pattern in file:", file_path)
                    matching_files.append(file_path)
        elif os.path.isdir(file_path):
            search_files_for_pattern(file_path, pattern)  # Recursively search subdirectories
    for i in range(len(matching_files)):
        print(matching_files[i])
    return matching_files

前端调用脚本 #

$.ajax({
    url: '{% url "process_form" %}',
    type: 'POST',
    data: jsonData,
    cache: false,
    contentType: false,
    processData: false,
    xhr:function(){
        const xhr = new window.XMLHttpRequest();
        xhr.upload.addEventListener('progress', e=>{
            if(e.lengthComputable){
                const percentProgress = (e.loaded/e.total)*100;
                console.log(percentProgress);
                progress_bar.innerHTML = `<div class="progress-bar progress-bar-striped bg-success"
        role="progressbar" style="width: ${percentProgress}%" aria-valuenow="${percentProgress}" aria-valuemin="0"
        aria-valuemax="100"></div>`
            }
        });
        return xhr
    },
    success: function(jsonData) {
        // Extract the pk value from the server's response
        var pk = jsonData.pk;
        const matchingFiles = jsonData.matching_files;
        // Check if pk is available before redirecting
        console.log("pk value (function):", jsonData.pk);
        if (pk) {
            // Construct the URL with the pk and redirect to the result_page
            var resultPageUrl = '{% url "result_page" %}' + pk + '/';  // Update the URL according to your URL patterns
            // Pass the matching_files list as a parameter in the URL
            resultPageUrl += '?matching_files=' + JSON.stringify(matchingFiles);
            window.location.href = resultPageUrl;
        } else {
            alert('Error occurred during upload. Please try again.');
            $('#upload_btn').prop('disabled', false);
            $('.progress').hide();
        }
    },
    error: function() {
        alert('Error occurred during upload. Please try again.');
        $('#upload_btn').prop('disabled', false);
    }
});

核心问题定位 #

递归函数search_files_for_pattern存在逻辑漏洞：当遍历到子目录时，仅调用了递归函数但未将递归返回的匹配结果合并到当前的matching_files列表中，导致子目录下的匹配文件完全丢失。如果当前目录没有匹配项，最终就会返回空列表。

修复方案 #

修改递归函数，将子目录递归返回的结果通过extend方法合并到主列表中：

修复后的递归函数 #

def search_files_for_pattern(directory, pattern):
    matching_files = list()
    for filename in os.listdir(directory):
        file_path = os.path.join(directory, filename)
        if os.path.isfile(file_path):
            with open(file_path, 'r', encoding='utf-8', errors='ignore') as file:
                file_contents = file.read()
                if re.search(pattern, file_contents):
                    matching_files.append(file_path)
        elif os.path.isdir(file_path):
            # 关键修改：接收递归返回的结果并合并到当前列表
            sub_dir_matches = search_files_for_pattern(file_path, pattern)
            matching_files.extend(sub_dir_matches)
    for path in matching_files:
        print(path)
    return matching_files

额外优化建议 #

1. 用os.walk简化递归逻辑：
   手动递归容易出错，os.walk可以自动遍历所有子目录，代码更简洁：

   def search_files_for_pattern(directory, pattern):
       matching_files = []
       for root, _, files in os.walk(directory):
           for file in files:
               file_path = os.path.join(root, file)
               with open(file_path, 'r', encoding='utf-8', errors='ignore') as f:
                   if re.search(pattern, f.read()):
                       matching_files.append(file_path)
       return matching_files
   

2. 大文件优化：逐行读取：
   一次性读取大文件会占用过多内存，改为逐行读取，找到匹配后立即停止：

   def search_files_for_pattern(directory, pattern):
       matching_files = []
       for root, _, files in os.walk(directory):
           for file in files:
               file_path = os.path.join(root, file)
               with open(file_path, 'r', encoding='utf-8', errors='ignore') as f:
                   for line in f:
                       if re.search(pattern, line):
                           matching_files.append(file_path)
                           break  # 找到匹配后终止当前文件读取
       return matching_files
   

3. 前端传参优化：
   当前通过URL传递matching_files的JSON字符串，可能遇到URL长度限制。建议改为在result_page视图中，通过pk从数据库查询关联数据，或直接从Django的session中读取该列表。

内容的提问来源于stack exchange，提问作者Shankar Kumar Yelapu