Matlab生成SHA-1哈希密码时字符串组合循环耗时过长求助

Hey there! Let's fix that brutal performance bottleneck in your password cracking code. The core issue with your current implementation is dynamic array expansion paired with nested loops that force MATLAB to do repeated memory reallocations—this is why it's taking days to run. Here's how to speed it up drastically:

Key Problems in Your Original Code #

• Every time you run y(k,1)=strcat(S(i),S(j)), MATLAB has to check if y has enough space. If not, it allocates a larger array, copies all existing data into it, then adds the new element. Doing this n² times (where n is your dictionary size) is exponentially slow for large dictionaries.
• Nested loops in MATLAB are rarely the fastest approach for vectorizable operations like generating all pairwise combinations.

Optimized Solution #

We'll use preallocation (to avoid memory reallocations) and vectorized operations (to replace nested loops with MATLAB's optimized built-in functions):

clc; clear all; close all;

% Load and clean the dictionary
fileID = fopen('H:\dictionary.txt');
S = textscan(fileID, '%s', 'Delimiter', '\n');
fclose(fileID);
S = S{1};
S = S(~cellfun('isempty', S)); % Remove empty entries
n = length(S);

% Preallocate output array (string array is faster than cell array for MATLAB R2016b+)
total_combinations = n * n;
y = strings(total_combinations, 1);

% Generate all pairwise combinations in one vectorized step
% Create grid of indices for all i,j pairs
[i_indices, j_indices] = meshgrid(1:n);
% Flatten indices to column vectors and concatenate strings
y = strcat(S(i_indices(:)), S(j_indices(:)));

Alternative Vectorized Approach (Using Repmat) #

If you prefer, you can use repmat to create repeated versions of your dictionary and concatenate in bulk:

% Create column-wise and row-wise repeats of the dictionary
S_col = repmat(S, n, 1);
S_row = repmat(S', 1, n);
% Flatten and concatenate
y = strcat(S_col(:), S_row(:));

Why This Works #

1. Preallocation: We calculate the total number of combinations upfront (n*n) and allocate the full size of y immediately. This eliminates all the slow memory reallocations from your original loop.
2. Vectorization: MATLAB's built-in functions like meshgrid and strcat (when used on arrays) are optimized to run in C-level code under the hood—way faster than interpreted nested loops.

Handling Extra Large Dictionaries #

If your dictionary is so big that n*n combinations exceed your available memory, split the process into chunks. For example, process 1000 entries from S at a time:

chunk_size = 1000;
for i_start = 1:chunk_size:n
    i_end = min(i_start + chunk_size - 1, n);
    current_S = S(i_start:i_end);
    % Generate combinations for this chunk
    chunk_combinations = strcat(repmat(current_S, n, 1), repmat(S', length(current_S), 1));
    % Save chunk to disk or process immediately instead of storing all in memory
    writematrix(chunk_combinations, ['combinations_chunk_', num2str(i_start), '.txt']);
end

This way, you avoid loading all combinations into memory at once.

内容的提问来源于stack exchange，提问作者Yash Shah