TEST OVERVIEW & EQUATIONS
The Welch t-test evaluates whether two independent group means differ when variances may be unequal.
Test statistic: $$t = \frac{\bar{x}_1 - \bar{x}_2 - \Delta_0}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}}$$
Confidence interval: $$(\bar{x}_1 - \bar{x}_2) \pm t_{\alpha/2,\,\nu} \sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}$$
Additional Notes
As degrees of freedom increase, Welch's t behaves like a z-test. When samples are small or variances highly imbalanced, keep the heavier tails in mind and report the estimated power.
MARKETING SCENARIOS
You're leading a marketing analytics project for an e-commerce brand that is A/B testing email campaigns. Group 1 might represent subscribers who received an optimized subject line, while Group 2 saw the current control subject line. After the campaign, you collected key performance indicators such as click-through rate, average order value, or downstream revenue per recipient.
Use the tool below to plug in the sample statistics for each variant, evaluate whether the observed lift is meaningful, and decide if the new creative warrants a broader rollout. Adjust the hypothesized difference to model the minimum effect size that would justify marketing spend.
INPUTS & SETTINGS
Select Data Entry / Upload Mode
Enter aggregated statistics for each group. Scenarios and uploads overwrite these fields automatically.
| Group | Name | Mean (μ) | Std. deviation (s) | Sample size (n) |
|---|---|---|---|---|
| Group 1 | ||||
| Group 2 |
Upload summary inputs
Provide a CSV/TSV with exactly two rows, including columns such as group,mean,sd,n. Add optional delta0 and alpha columns to set the hypothesized difference and confidence level along with the stats.
Drag & drop summary file
Two groups required. Each row should describe one group.
No summary file uploaded.
Upload raw data
Upload a long-format file with columns like group,value. We’ll calculate the means, standard deviations, and sample sizes for the first two groups discovered.
Drag & Drop raw data file (.csv, .tsv, .txt)
Long format with headers like group,value; at least two unique groups. Up to 2,000 rows.
No raw file uploaded.
Analysis Settings
VISUAL OUTPUT
Difference Fan Chart
Visual Output Settings
Means Fan Chart
Difference Fan Chart
TEST RESULTS
APA-Style Statistical Reporting
Managerial Interpretation
Summary of Estimates
| Measure | Estimate | Standard Error | Lower Bound | Upper Bound |
|---|
DIAGNOSTICS & ASSUMPTIONS
Diagnostics & Assumption Tests
Provide group summaries to review sample size balance, variance ratios, power, and interval diagnostics.