{"id":1059,"date":"2026-05-12T13:00:00","date_gmt":"2026-05-12T18:00:00","guid":{"rendered":"https:\/\/tolinku.com\/blog\/?p=1059"},"modified":"2026-03-07T03:34:29","modified_gmt":"2026-03-07T08:34:29","slug":"multivariate-testing-deep-links","status":"publish","type":"post","link":"https:\/\/tolinku.com\/blog\/multivariate-testing-deep-links\/","title":{"rendered":"Multivariate Testing for Deep Link Campaigns"},"content":{"rendered":"\n

A\/B testing compares two variants of one variable. Multivariate testing (MVT) tests multiple variables at the same time, showing you not just which variant wins but which combination of variables produces the best result. For deep link campaigns, this means testing CTA text, landing page layout, and destination screen simultaneously instead of running three separate sequential tests.<\/p>\n\n\n\n

For A\/B testing fundamentals, see A\/B Testing Deep Links and Landing Pages<\/a>. For sample size planning, see A\/B Testing Sample Size Calculator for Deep Links<\/a>.<\/p>\n\n\n\n

\"Tolinku\nThe A\/B tests list page showing test names, status, types, and variant counts.<\/em><\/p>\n\n\n\n

A\/B Testing vs. Multivariate Testing<\/h2>\n\n\n\n
\n\n\n\n\n\n\n\n\n\n
Aspect<\/th>\nA\/B Testing<\/th>\nMultivariate Testing<\/th>\n<\/tr>\n<\/thead>\n
Variables tested<\/td>\n1<\/td>\n2+ simultaneously<\/td>\n<\/tr>\n
Variants<\/td>\n2-4<\/td>\nCan be dozens (factorial)<\/td>\n<\/tr>\n
Traffic required<\/td>\nModerate<\/td>\nHigh<\/td>\n<\/tr>\n
Time to results<\/td>\nWeeks<\/td>\nWeeks to months<\/td>\n<\/tr>\n
Insights<\/td>\n"B beats A"<\/td>\n"This combination is optimal"<\/td>\n<\/tr>\n
Interaction effects<\/td>\nNot detected<\/td>\nDetected<\/td>\n<\/tr>\n
Complexity<\/td>\nLow<\/td>\nMedium to high<\/td>\n<\/tr>\n<\/tbody><\/table><\/figure>\n\n\n\n

When to Use MVT<\/h3>\n\n\n\n

Use multivariate testing when:<\/p>\n\n\n\n

    \n
  • You have high traffic<\/strong> (10,000+ daily impressions on the tested surface)<\/li>\n
  • You suspect interaction effects<\/strong> between variables (e.g., a certain headline only works with a specific CTA)<\/li>\n
  • You want to optimize multiple elements at once<\/strong> instead of running sequential A\/B tests<\/li>\n
  • The elements being tested are on the same page or surface<\/strong><\/li>\n<\/ul>\n\n\n\n

    Stick with A\/B testing when:<\/p>\n\n\n\n

      \n
    • Traffic is low (< 5,000 daily impressions)<\/li>\n
    • You're testing a single major change (new design vs. old)<\/li>\n
    • You need results quickly<\/li>\n<\/ul>\n\n\n\n

      How MVT Works<\/h2>\n\n\n\n

      Full Factorial Design<\/h3>\n\n\n\n

      Test every combination of every variable level:<\/p>\n\n\n\n

      function generateFullFactorial(variables) {\n  \/\/ variables = { headline: ['A', 'B'], cta: ['X', 'Y', 'Z'], layout: ['short', 'long'] }\n  const keys = Object.keys(variables);\n  const combinations = [];\n\n  function combine(index, current) {\n    if (index === keys.length) {\n      combinations.push({ ...current });\n      return;\n    }\n\n    const key = keys[index];\n    for (const value of variables[key]) {\n      current[key] = value;\n      combine(index + 1, current);\n    }\n  }\n\n  combine(0, {});\n  return combinations;\n}\n\n\/\/ Example\nconst variants = generateFullFactorial({\n  headline: ['benefit', 'feature'],\n  cta: ['download', 'try_free', 'get_started'],\n  layout: ['short', 'long'],\n});\n\nconsole.log(variants.length); \/\/ 2 x 3 x 2 = 12 combinations\n<\/code><\/pre>\n\n\n\n
      Fractional Factorial Design<\/h3>\n\n\n\n
      When full factorial requires too many combinations, test a strategic subset:<\/p>\n\n\n\n
      function generateFractionalFactorial(variables, fraction) {\n  const full = generateFullFactorial(variables);\n\n  \/\/ Use orthogonal array to select a balanced subset\n  \/\/ For a 1\/2 fraction: test half the combinations\n  \/\/ while maintaining ability to estimate main effects\n  const selected = selectOrthogonalSubset(full, fraction);\n\n  return {\n    combinations: selected,\n    fullSize: full.length,\n    fractionSize: selected.length,\n    canEstimate: 'main effects + some interactions',\n    cannotEstimate: 'higher-order interactions',\n  };\n}\n\n\/\/ A Taguchi L8 array for testing 7 two-level factors in only 8 runs\nconst taguchiL8 = [\n  { headline: 'A', cta: 'X', layout: 'short', image: 'yes', badge: 'yes', social: 'count', urgency: 'no' },\n  { headline: 'A', cta: 'X', layout: 'short', image: 'no', badge: 'no', social: 'rating', urgency: 'yes' },\n  { headline: 'A', cta: 'Y', layout: 'long', image: 'yes', badge: 'yes', social: 'rating', urgency: 'yes' },\n  { headline: 'A', cta: 'Y', layout: 'long', image: 'no', badge: 'no', social: 'count', urgency: 'no' },\n  { headline: 'B', cta: 'X', layout: 'long', image: 'yes', badge: 'no', social: 'count', urgency: 'yes' },\n  { headline: 'B', cta: 'X', layout: 'long', image: 'no', badge: 'yes', social: 'rating', urgency: 'no' },\n  { headline: 'B', cta: 'Y', layout: 'short', image: 'yes', badge: 'no', social: 'rating', urgency: 'no' },\n  { headline: 'B', cta: 'Y', layout: 'short', image: 'no', badge: 'yes', social: 'count', urgency: 'yes' },\n];\n<\/code><\/pre>\n\n\n\n
      Implementation<\/h2>\n\n\n\n
      MVT Configuration<\/h3>\n\n\n\n
      const mvtExperiment = {\n  id: 'landing_page_mvt_v1',\n  type: 'multivariate',\n  variables: {\n    headline: {\n      levels: [\n        { id: 'benefit', value: 'Save 2 Hours Every Week' },\n        { id: 'social', value: 'Join 500K+ Teams' },\n      ],\n    },\n    cta: {\n      levels: [\n        { id: 'download', value: 'Download Free' },\n        { id: 'try', value: 'Try It Free' },\n        { id: 'start', value: 'Get Started' },\n      ],\n    },\n    socialProof: {\n      levels: [\n        { id: 'rating', value: { type: 'stars', rating: 4.8 } },\n        { id: 'count', value: { type: 'userCount', count: '500K+' } },\n      ],\n    },\n  },\n  design: 'full_factorial', \/\/ 2 x 3 x 2 = 12 combinations\n  primaryMetric: 'app_install',\n  minSamplePerCombination: 500,\n  \/\/ Total required: 12 x 500 = 6,000\n};\n<\/code><\/pre>\n\n\n\n
      Variant Assignment<\/h3>\n\n\n\n
      function assignMVTVariant(userId, experiment) {\n  const existing = storage.get(`mvt_${experiment.id}_${userId}`);\n  if (existing) return existing;\n\n  \/\/ Generate all combinations\n  const combinations = generateFullFactorial(\n    Object.fromEntries(\n      Object.entries(experiment.variables).map(([key, config]) => [\n        key,\n        config.levels.map(l => l.id),\n      ])\n    )\n  );\n\n  \/\/ Deterministic assignment\n  const hash = djb2Hash(`${userId}-${experiment.id}`);\n  const index = hash % combinations.length;\n  const assignment = combinations[index];\n\n  \/\/ Resolve to actual values\n  const resolved = {};\n  for (const [variable, levelId] of Object.entries(assignment)) {\n    const level = experiment.variables[variable].levels.find(l => l.id === levelId);\n    resolved[variable] = level.value;\n  }\n\n  const variant = {\n    combinationIndex: index,\n    assignment,\n    resolved,\n  };\n\n  storage.set(`mvt_${experiment.id}_${userId}`, variant);\n  return variant;\n}\n<\/code><\/pre>\n\n\n\n
      Rendering the Page<\/h3>\n\n\n\n
      function LandingPage({ experiment }) {\n  const variant = useMVTVariant(experiment);\n\n  return (\n    <Page>\n      <Hero>\n        <Heading>{variant.resolved.headline}<\/Heading>\n        <CTAButton>{variant.resolved.cta}<\/CTAButton>\n      <\/Hero>\n\n      <SocialProof config={variant.resolved.socialProof} \/>\n\n      <TrackImpression\n        experimentId={experiment.id}\n        combination={variant.assignment}\n      \/>\n    <\/Page>\n  );\n}\n<\/code><\/pre>\n\n\n\n
      Analyzing MVT Results<\/h2>\n\n\n\n
      Main Effects<\/h3>\n\n\n\n
      The main effect of a variable is its average impact across all levels of other variables:<\/p>\n\n\n\n
      async function calculateMainEffects(experimentId) {\n  const experiment = getExperiment(experimentId);\n  const results = await getAllCombinationResults(experimentId);\n\n  const mainEffects = {};\n\n  for (const [variable, config] of Object.entries(experiment.variables)) {\n    mainEffects[variable] = {};\n\n    for (const level of config.levels) {\n      \/\/ Get all combinations where this variable = this level\n      const matching = results.filter(r => r.assignment[variable] === level.id);\n      const avgConversion = matching.reduce((sum, r) => sum + r.conversionRate, 0) \/ matching.length;\n\n      mainEffects[variable][level.id] = {\n        avgConversion: (avgConversion * 100).toFixed(2) + '%',\n        sampleSize: matching.reduce((sum, r) => sum + r.impressions, 0),\n      };\n    }\n  }\n\n  return mainEffects;\n}\n\n\/\/ Example output:\n\/\/ {\n\/\/   headline: { benefit: { avg: '4.2%' }, social: { avg: '3.8%' } },\n\/\/   cta: { download: { avg: '4.5%' }, try: { avg: '3.9%' }, start: { avg: '3.6%' } },\n\/\/   socialProof: { rating: { avg: '4.1%' }, count: { avg: '3.9%' } },\n\/\/ }\n<\/code><\/pre>\n\n\n\n
      Interaction Effects<\/h3>\n\n\n\n
      Interaction effects show when variables influence each other:<\/p>\n\n\n\n
      async function calculateInteractions(experimentId) {\n  const results = await getAllCombinationResults(experimentId);\n  const experiment = getExperiment(experimentId);\n  const variables = Object.keys(experiment.variables);\n\n  const interactions = [];\n\n  \/\/ Check all pairs of variables\n  for (let i = 0; i < variables.length; i++) {\n    for (let j = i + 1; j < variables.length; j++) {\n      const varA = variables[i];\n      const varB = variables[j];\n\n      \/\/ For each combination of levels\n      const levelsA = experiment.variables[varA].levels;\n      const levelsB = experiment.variables[varB].levels;\n\n      const grid = {};\n      for (const la of levelsA) {\n        grid[la.id] = {};\n        for (const lb of levelsB) {\n          const matching = results.filter(r =>\n            r.assignment[varA] === la.id && r.assignment[varB] === lb.id\n          );\n          grid[la.id][lb.id] = matching.reduce((sum, r) => sum + r.conversionRate, 0) \/ matching.length;\n        }\n      }\n\n      \/\/ Check if effect of varA differs by level of varB\n      const hasInteraction = checkInteraction(grid, levelsA, levelsB);\n      if (hasInteraction.significant) {\n        interactions.push({\n          variables: [varA, varB],\n          grid,\n          strength: hasInteraction.strength,\n        });\n      }\n    }\n  }\n\n  return interactions;\n}\n<\/code><\/pre>\n\n\n\n
      Example: An interaction exists when "Download Free" (CTA) works better with "Save 2 Hours" (headline) but "Try It Free" works better with "Join 500K+ Teams." Without MVT, you'd miss this.<\/p>\n\n\n\n
      Finding the Winning Combination<\/h3>\n\n\n\n
      async function findWinner(experimentId) {\n  const results = await getAllCombinationResults(experimentId);\n\n  \/\/ Sort by conversion rate\n  const sorted = results.sort((a, b) => b.conversionRate - a.conversionRate);\n\n  \/\/ Check if the top combination is significantly better than the others\n  const best = sorted[0];\n  const secondBest = sorted[1];\n\n  const significance = zTest(\n    secondBest.conversions, secondBest.impressions,\n    best.conversions, best.impressions\n  );\n\n  return {\n    winner: best.assignment,\n    conversionRate: (best.conversionRate * 100).toFixed(2) + '%',\n    lift: ((best.conversionRate - secondBest.conversionRate) \/ secondBest.conversionRate * 100).toFixed(1) + '%',\n    vsSecondBest: significance,\n    allResults: sorted.map(r => ({\n      combination: r.assignment,\n      rate: (r.conversionRate * 100).toFixed(2) + '%',\n      impressions: r.impressions,\n    })),\n  };\n}\n<\/code><\/pre>\n\n\n\n
      Traffic Requirements<\/h2>\n\n\n\n
      Full Factorial<\/h3>\n\n\n\n
      function mvtTrafficRequirement(variables, baselineRate, mde, perCombinationSample = 500) {\n  const numCombinations = Object.values(variables).reduce(\n    (product, levels) => product * levels.length,\n    1\n  );\n\n  const totalRequired = numCombinations * perCombinationSample;\n\n  return {\n    combinations: numCombinations,\n    perCombination: perCombinationSample,\n    totalRequired,\n    note: `Need ${totalRequired.toLocaleString()} total impressions`,\n  };\n}\n\n\/\/ 2 x 3 x 2 = 12 combinations x 500 each = 6,000 total\n\/\/ At 1000\/day = 6 days (but need 14 minimum for weekly patterns)\n<\/code><\/pre>\n\n\n\n
      Quick Reference<\/h3>\n\n\n\n
      \n\n\n\n\n\n\n\n\n
      Variables<\/th>\nLevels Each<\/th>\nCombinations<\/th>\nMin Traffic (500\/combo)<\/th>\n<\/tr>\n<\/thead>\n
      2 vars<\/td>\n2 each<\/td>\n4<\/td>\n2,000<\/td>\n<\/tr>\n
      2 vars<\/td>\n3 each<\/td>\n9<\/td>\n4,500<\/td>\n<\/tr>\n
      3 vars<\/td>\n2 each<\/td>\n8<\/td>\n4,000<\/td>\n<\/tr>\n
      3 vars<\/td>\n3 each<\/td>\n27<\/td>\n13,500<\/td>\n<\/tr>\n
      4 vars<\/td>\n2 each<\/td>\n16<\/td>\n8,000<\/td>\n<\/tr>\n
      4 vars<\/td>\n3 each<\/td>\n81<\/td>\n40,500<\/td>\n<\/tr>\n<\/tbody><\/table><\/figure>\n\n\n\n
      Beyond 3 variables with 3 levels each, consider fractional factorial design to reduce traffic requirements.<\/p>\n\n\n\n
      Deep Link MVT Scenarios<\/h2>\n\n\n\n
      Scenario 1: Landing Page Optimization<\/h3>\n\n\n\n
      Test headline, CTA, and social proof on the fallback landing page:<\/p>\n\n\n\n
      const landingPageMVT = {\n  id: 'landing_mvt_v1',\n  variables: {\n    headline: {\n      levels: [\n        { id: 'benefit', value: 'Save 2 Hours Every Week' },\n        { id: 'social', value: 'Join 500K+ Teams' },\n      ],\n    },\n    cta: {\n      levels: [\n        { id: 'download', value: 'Download Free' },\n        { id: 'try', value: 'Try It Free' },\n      ],\n    },\n    proof: {\n      levels: [\n        { id: 'stars', value: '4.8 stars on App Store' },\n        { id: 'users', value: 'Used by 500K+ teams' },\n      ],\n    },\n  },\n  \/\/ 2 x 2 x 2 = 8 combinations, manageable\n};\n<\/code><\/pre>\n\n\n\n
      Scenario 2: Smart Banner Configuration<\/h3>\n\n\n\n
      Test banner copy, position, and timing:<\/p>\n\n\n\n
      const bannerMVT = {\n  id: 'banner_mvt_v1',\n  variables: {\n    copy: {\n      levels: [\n        { id: 'benefit', value: 'Better experience in the app' },\n        { id: 'action', value: 'Continue in the app' },\n      ],\n    },\n    position: {\n      levels: [\n        { id: 'top', value: 'top' },\n        { id: 'bottom', value: 'bottom' },\n      ],\n    },\n    timing: {\n      levels: [\n        { id: 'immediate', value: 0 },\n        { id: 'delayed', value: 3000 },\n      ],\n    },\n  },\n  \/\/ 2 x 2 x 2 = 8 combinations\n};\n<\/code><\/pre>\n\n\n\n
      Scenario 3: Email Campaign Elements<\/h3>\n\n\n\n
      Test subject line, CTA, and link destination:<\/p>\n\n\n\n
      const emailMVT = {\n  id: 'email_mvt_v1',\n  variables: {\n    subject: {\n      levels: [\n        { id: 'direct', value: 'Your order is ready' },\n        { id: 'app', value: 'Your order is ready - track in app' },\n      ],\n    },\n    cta: {\n      levels: [\n        { id: 'generic', value: 'View Order' },\n        { id: 'specific', value: 'Track Delivery' },\n      ],\n    },\n    destination: {\n      levels: [\n        { id: 'order_page', value: '\/orders\/:id' },\n        { id: 'tracking', value: '\/orders\/:id\/tracking' },\n      ],\n    },\n  },\n  \/\/ 2 x 2 x 2 = 8 combinations\n};\n<\/code><\/pre>\n\n\n\n
      Best Practices<\/h2>\n\n\n\n
        \n
      1. Limit to 2-3 variables<\/strong>: More variables means exponentially more combinations and traffic.<\/li>\n
      2. Use 2 levels per variable when possible<\/strong>: Going from 2 to 3 levels multiplies combinations by 1.5x per variable.<\/li>\n
      3. Run for at least 14 days<\/strong>: Weekly patterns affect all combinations.<\/li>\n
      4. Check for interactions<\/strong>: The main value of MVT over sequential A\/B tests is discovering interactions. Don't skip this analysis.<\/li>\n
      5. Start with A\/B, graduate to MVT<\/strong>: Run A\/B tests first to identify which variables matter, then use MVT to find the optimal combination.<\/li>\n
      6. Consider fractional factorial for high-variable tests<\/strong>: You don't always need every combination.<\/li>\n<\/ol>\n\n\n\n
        For A\/B testing features, see Tolinku A\/B testing<\/a>. For test setup, see the A\/B testing docs<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"
        Go beyond simple A\/B testing with multivariate tests. Test multiple variables simultaneously to find the optimal deep link configuration faster.<\/p>\n","protected":false},"author":2,"featured_media":1058,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"rank_math_title":"Multivariate Testing for Deep Link Campaigns","rank_math_description":"Go beyond simple A\/B testing with multivariate tests. Test multiple variables simultaneously to find optimal deep link configurations.","rank_math_focus_keyword":"multivariate testing deep links","rank_math_canonical_url":"","rank_math_facebook_title":"","rank_math_facebook_description":"","rank_math_facebook_image":"https:\/\/tolinku.com\/blog\/wp-content\/uploads\/2026\/03\/og-multivariate-testing-deep-links.png","rank_math_facebook_image_id":"","rank_math_twitter_title":"","rank_math_twitter_description":"","rank_math_twitter_image":"https:\/\/tolinku.com\/blog\/wp-content\/uploads\/2026\/03\/og-multivariate-testing-deep-links.png","footnotes":""},"categories":[13],"tags":[60,37,191,20,225,69,256,258],"class_list":["post-1059","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-growth","tag-ab-testing","tag-analytics","tag-conversions","tag-deep-linking","tag-experimentation","tag-mobile-development","tag-optimization","tag-statistics"],"_links":{"self":[{"href":"https:\/\/tolinku.com\/blog\/wp-json\/wp\/v2\/posts\/1059","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tolinku.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tolinku.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tolinku.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/tolinku.com\/blog\/wp-json\/wp\/v2\/comments?post=1059"}],"version-history":[{"count":2,"href":"https:\/\/tolinku.com\/blog\/wp-json\/wp\/v2\/posts\/1059\/revisions"}],"predecessor-version":[{"id":2231,"href":"https:\/\/tolinku.com\/blog\/wp-json\/wp\/v2\/posts\/1059\/revisions\/2231"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/tolinku.com\/blog\/wp-json\/wp\/v2\/media\/1058"}],"wp:attachment":[{"href":"https:\/\/tolinku.com\/blog\/wp-json\/wp\/v2\/media?parent=1059"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tolinku.com\/blog\/wp-json\/wp\/v2\/categories?post=1059"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tolinku.com\/blog\/wp-json\/wp\/v2\/tags?post=1059"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}