Firebase A/B测试累计事件量统计方法咨询及新菜单测试矛盾问题

Hey there, let's break down your questions based on what you're seeing with your new card-style menu and A/B test results.

1. Does Firebase A/B Test only measure if a user had at least one ad_impression? #

Yep, that's exactly the default behavior. When you set an event like ad_impression as your test target, Firebase's default metric counts the percentage of users who triggered the event at least once—not the total number of times the event fired across all users.

That explains the discrepancy you're seeing: the console shows worse ad_impression performance because it's tracking how many users had any ad impression, but your user-level stats reveal better revenue. Chances are your card-style menu leads to fewer impressions per user, but each impression drives higher revenue (better click-through rates, higher-value ads, etc.). So the "user conversion count" metric isn't capturing the full picture here.

2. How to test the total number of event occurrences? #

To measure the cumulative number of ad_impression events, you need to switch from the default user-centric metric to an event-count metric when setting up your test target. Here's how:

• When creating or editing your A/B test in the Firebase Console, after selecting ad_impression as your target event, look for the metric options. Instead of choosing the default "Event count" (which tracks users who triggered the event), select "Total events" (or sometimes labeled "Sum of events")—this will tally up every single occurrence of ad_impression across all users in each variant.

For even more control, you can add a custom parameter to your ad_impression event:

• Every time you log ad_impression, include a parameter like impression_value set to 1.
• Then, in your test target, select "Sum of event parameter" and pick impression_value—this will give you the exact total number of impressions, which works identically to the "Total events" metric but lets you extend this logic to other numeric parameters if needed (like revenue per impression).

Also, don't forget to cross-reference with other metrics in Firebase Analytics: you can create custom reports that segment by your A/B test variant and show total ad_impression counts alongside revenue metrics, to get a complete view of how your new menu is performing.

内容的提问来源于stack exchange，提问作者Mateusz Kaflowski