TensorFlow 1.5.1模型C++端从.meta/.ckpt加载并推理实现方法
Got it, let's walk through how to get your TensorFlow 1.5.1 model (from .ckpt and .meta files) running in C++. I've done this a few times, so here are the two main approaches you can take—one that uses your existing checkpoints directly, and another that converts to the more portable SavedModel format first.
First, make sure you have the TensorFlow 1.5.1 C++ library set up on your system. You can either build it from source (matching the 1.5.1 tag) or use a precompiled package. Just note that TensorFlow 2.x libraries won't work here—stick strictly to 1.x for this version.
This approach uses your existing files without conversion. The key steps are loading the graph structure from the .meta file, restoring variable values from the .ckpt, and then running inference.
Step 1: Confirm Input/Output Node Names in Python
Before writing C++ code, you need to know the exact names of your input and output tensors. In your Python inference code, add these lines to print them:
# After loading your model in Python print("Input tensor name:", input_tensor.name) print("Output tensor name:", output_tensor.name)
You'll get something like input:0 or dense_1/BiasAdd:0—keep these handy, you'll need them in C++.
Step 2: C++ Inference Code
Here's a complete example that loads the graph, restores variables, and runs inference:
#include <tensorflow/core/public/session.h> #include <tensorflow/core/platform/env.h> #include <tensorflow/core/framework/tensor.h> #include <iostream> using namespace tensorflow; int main() { // 1. Initialize session options SessionOptions session_options; std::unique_ptr<Session> session(NewSession(session_options)); // 2. Load graph structure from .meta file GraphDef graph_def; Status load_status = ReadBinaryProto(Env::Default(), "/path/to/your/model.meta", &graph_def); if (!load_status.ok()) { std::cerr << "Failed to load graph definition: " << load_status.ToString() << std::endl; return 1; } // 3. Create session with the loaded graph Status create_status = session->Create(graph_def); if (!create_status.ok()) { std::cerr << "Failed to create session: " << create_status.ToString() << std::endl; return 1; } // 4. Restore variables from .ckpt (use the base name, no .data-* suffix) Tensor checkpoint_path(DT_STRING, TensorShape()); checkpoint_path.scalar<std::string>()() = "/path/to/your/model.ckpt"; Status restore_status = session->Run( {{"save/Const:0", checkpoint_path}}, // Input to restore op {}, // No outputs needed for restore {"save/restore_all"}, // Restore all variables nullptr ); if (!restore_status.ok()) { std::cerr << "Failed to restore variables: " << restore_status.ToString() << std::endl; return 1; } // 5. Prepare input data (match shape/dtype from training) // Example: Input is a 1x224x224x3 float tensor (adjust to your model's input) Tensor input_tensor(DT_FLOAT, TensorShape({1, 224, 224, 3})); auto input_flat = input_tensor.flat<float>(); // Fill input data here (e.g., preprocess an image and copy values) // for (int i = 0; i < input_flat.size(); ++i) { input_flat(i) = your_preprocessed_data[i]; } // 6. Run inference (replace with your input/output node names) std::vector<Tensor> outputs; Status run_status = session->Run( {{"input:0", input_tensor}}, // Input tensor name from Python {"output:0"}, // Output tensor name from Python {}, &outputs ); if (!run_status.ok()) { std::cerr << "Inference failed: " << run_status.ToString() << std::endl; return 1; } // 7. Process and print results Tensor& output_tensor = outputs[0]; auto output_flat = output_tensor.flat<float>(); for (int i = 0; i < output_flat.size(); ++i) { std::cout << "Output[" << i << "]: " << output_flat(i) << std::endl; } // Cleanup session->Close(); return 0; }
Step 3: Compile the Code
Use a command like this (adjust paths if your TensorFlow library is in a non-standard location):
g++ -std=c++11 inference.cpp -o inference -ltensorflow
SavedModel is TensorFlow's cross-language standard format, which bundles the graph and variables into a single directory. This makes loading in C++ cleaner and less error-prone.
Step 1: Convert Model in Python
Run this code to convert your .ckpt/.meta to SavedModel:
import tensorflow as tf # Load your existing model sess = tf.Session() saver = tf.train.import_meta_graph('/path/to/your/model.meta') saver.restore(sess, '/path/to/your/model.ckpt') # Get your input/output tensors (use your existing collection or direct references) input_tensor = tf.get_collection('input')[0] output_tensor = tf.get_collection('output')[0] # Build SavedModel builder = tf.saved_model.builder.SavedModelBuilder('/path/to/saved_model_dir') # Define serving signature (maps input/output names for inference) signature = tf.saved_model.signature_def_utils.build_signature_def( inputs={'model_input': tf.saved_model.utils.build_tensor_info(input_tensor)}, outputs={'model_output': tf.saved_model.utils.build_tensor_info(output_tensor)}, method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME ) # Save the model builder.add_meta_graph_and_variables( sess, [tf.saved_model.tag_constants.SERVING], signature_def_map={ tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: signature } ) builder.save()
Step 2: C++ Code to Load SavedModel
This code is simpler since SavedModel handles graph and variable loading in one step:
#include <tensorflow/core/public/session.h> #include <tensorflow/core/platform/env.h> #include <tensorflow/core/protobuf/meta_graph.pb.h> #include <iostream> using namespace tensorflow; int main() { SessionOptions options; std::unique_ptr<Session> session(NewSession(options)); // Load SavedModel's meta graph MetaGraphDef meta_graph_def; Status load_status = ReadBinaryProto( Env::Default(), "/path/to/saved_model_dir/saved_model.pb", &meta_graph_def ); if (!load_status.ok()) { std::cerr << "Failed to load meta graph: " << load_status.ToString() << std::endl; return 1; } // Create session with the graph Status create_status = session->Create(meta_graph_def.graph_def()); if (!create_status.ok()) { std::cerr << "Failed to create session: " << create_status.ToString() << std::endl; return 1; } // Prepare input data (same as method 1) Tensor input_tensor(DT_FLOAT, TensorShape({1, 224, 224, 3})); auto input_flat = input_tensor.flat<float>(); // Fill input data here // Run inference (use either the signature name or tensor node name) std::vector<Tensor> outputs; Status run_status = session->Run( {{"input:0", input_tensor}}, // Or use "model_input" from the signature {"output:0"}, // Or use "model_output" {}, &outputs ); if (!run_status.ok()) { std::cerr << "Inference failed: " << run_status.ToString() << std::endl; return 1; } // Process results Tensor& output_tensor = outputs[0]; auto output_flat = output_tensor.flat<float>(); for (int i = 0; i < output_flat.size(); ++i) { std::cout << "Output[" << i << "]: " << output_flat(i) << std::endl; } session->Close(); return 0; }
- Node Name Matching: Double-check that the input/output node names in C++ exactly match what you printed in Python (including the
:0suffix, which refers to the first tensor of the node). - Data Type & Shape: Ensure your C++ input tensor has the same dtype (e.g.,
DT_FLOATforfloat32) and shape as the Python model expects. Mismatches will cause runtime errors. - TensorFlow Version: Use the exact 1.5.1 C++ library—mixing versions (e.g., Python 1.5.1 with C++ 1.15) will lead to compatibility issues.
- GPU Support: If you're using a GPU model, make sure your C++ library is compiled with CUDA support, and set the appropriate environment variables (e.g.,
CUDA_VISIBLE_DEVICES).
内容的提问来源于stack exchange，提问作者a.ewais
