top of page
Writer's pictureLee Gang

Calculating ROI for Gen AI Projects

Updated: Oct 10, 2023

Estimating the Return on Investment (ROI) for AI/ML projects, especially Generative AI (Gen AI), presents unique challenges due to inherent uncertainties and intangible benefits. This article offers a structured framework, detailing eight essential steps, to guide businesses in effectively calculating the potential ROI for such projects, ensuring they make informed decisions before implementation.


Balancing costs and benefit for AI Projects

Estimating ROI On High Uncertainty AI Projects


Navigating the intricate world of AI/ML projects, many organizations grapple with predicting their Return on Investment (ROI). Unlike conventional projects where assessing risks and returns is more straightforward, AI and specifically, Generative AI (Gen AI) projects, confront higher uncertainties (for a non-technical primer on Gen AI, see this post). These span from fluctuating costs to the myriad intangible benefits that can be challenging to quantify.


Recognizing ROI is paramount before plunging into the depths of any project. There are companies that jump in too hastily, only to later realize that their ROI isn't up to par. Conversely, others might miss great opportunities, being overly cautious and seeking pinpoint ROI accuracy before initiation. This article serves as a simple guide (with examples), offering a structured approach to evaluate the ROI of Gen AI and broadly, any AI/ML project.


Step 1: Set Crystal Clear Objectives


Every successful project commences with defined objectives. In the AI realm, these can range from operational efficiency enhancements and better customer engagement to product innovations or workforce development. Pinpoint a singular, primary objective. Spreading oneself too thin by setting multiple primary goals can dilute the project's potency. While secondary objectives are permissible, the primary one should take precedence.


Example: Company A wanted to leverage on Gen AI to identify and reach out to potential leads via hyper-personalized emails. Company A decides that its primary objective would be to increase the number of leads from its main targeted customer segment.


Step 2: Pinpoint Key Metrics


Once your objectives are in place, settle on the metrics that will chart your project's trajectory. This could be anything from cost savings, revenue upticks, customer contentment levels, or other pertinent performance yardsticks. Ensure these metrics resonate with your goals and are quantifiable.


Example: Company A decides to monitor:

  • Percentage Uplift in Qualified Sales Lead: A measure of growth in leads from targeted customer brackets.

  • Time To Value: The duration between initiating an investment and the realization of its value.


Step 3: Project Costs


Now, delineate the financial implications of your endeavour. This encompasses the upfront investment for kickstarting the project and the consistent costs that follow. Recognizing the potential variability in costs, create different scenarios like 'ideal', 'most likely', and 'worst case' to accommodate uncertainties.

  • One-Time Costs: These encompass software/hardware purchases, talent acquisition, data gathering, training, and execution costs.

  • Recurring Costs: These pertain to subscription/licensing fees, upkeep, and hosting charges.

Example: Company A lists down its cost estimates for each of the scenarios:

  • Ideal Scenario (20% Probability of Happening):

    1. Implementation (One-Time): $30,000

    2. Data Acquisition & Preparation (One-Time): $10,000

    3. Cloud Hosting (Monthly): $1,000

    4. Service & Maintenance (Monthly): $2,000

    5. Total One-Time Costs: $40,000

    6. Total Ongoing Costs: $3,000

  • Most Likely Scenario (60% Probability of Happening):

    1. Implementation (One-Time): $45,000

    2. Data Acquisition & Preparation (One-Time): $10,000

    3. Cloud Hosting (Monthly): $1,000

    4. Service & Maintenance (Monthly): $2,000

    5. Total One-Time Costs: $55,000

    6. Total Ongoing Costs: $3,000

  • Worst Case Scenario (20% Probability of Happening):

    1. Implementation (One-Time): $60,000

    2. Data Acquisition & Preparation (One-Time): $10,000

    3. Cloud Hosting (Monthly): $1,000

    4. Service & Maintenance (Monthly): $2,000

    5. Total One-Time Costs: $70,000

    6. Total Ongoing Costs: $3,000


Step 4: Gauge Benefits


To estimate the benefits of your project, consider the potential impact on the key metrics identified in Step 2. This could include factors such as increased revenue, cost savings, improved customer satisfaction, or productivity gains. Quantify these benefits in measurable terms and assign a value to each one. It is important to base these estimates on realistic assumptions and data to ensure accuracy. Intangible benefits such as improved employee satisfaction, operational flexibility, and brand enhancement might be difficult to quantify, but you can assign a relative importance to each or rank them.


Example: Company A estimates the benefits of its Gen AI project under different scenarios:

  • Ideal Scenario:

    1. Increase in qualified sales leads: 50% (which translate to 10% increase in sales, $100,000 annually)

    2. Time to value: 6 months

    3. (Intangible) Better employee satisfaction due to reduced cold outreach.

    4. (Intangible) Improved outreach to potentially viable but untested segments.

  • Most Likely Scenario:

    1. Increase in qualified sales leads: 30% (which translate to 6% increase in sales, $60,000 annually)

    2. Time to value: 9 months

    3. (Intangible) Better employee satisfaction due to reduced cold outreach.

    4. (Intangible) Improved outreach to potentially viable but untested segments.

  • Worst Case Scenario:

    1. Increase in qualified sales leads: 20% (which translate to 4% increase in sales, $40,000 annually)

    2. Time to value: 12 months

    3. (Intangible) Better employee satisfaction due to reduced cold outreach.

    4. (Intangible) Improved outreach to potentially viable but untested segments.


Step 5: Consolidate Costs & Benefits for Each Scenario


Establish a project lifecycle (commonly 3-5 years) and then compute the Total Benefits and Total Costs for each scenario.

  • Total Benefits = One-Benefits + Ongoing Benefits x Project Lifecycle

  • Total Costs = One-Time Costs + Ongoing Costs x Project Lifecycle

Example: Company A calculates the Total Costs and Total Benefits for the project under different scenarios:

  • Ideal Scenario:

    1. Total Benefits: $100,000 x 3 = $300,000

    2. Total Costs: $40,000 + $3,000 x 36 = $148,000

  • Most Likely Scenario:

    1. Total Benefits: $60,000 x 3 = $180,000

    2. Total Costs: $55,000 + $3,000 x 36 = $163,000

  • Worst Case Scenario:

    1. Total Benefits: $40,000 x 3 = $120,000

    2. Total Costs: $70,000 + $3,000 x 36 = $178,000


Step 6: Derive Expected ROI


Now, assign the probability of each scenario happening, and combine the Total Costs and Total Benefits for each scenario into Expected Total Costs and Expected Total Benefits respectively. Finally, you can calculate the Expected ROI.

  • Expected Total Costs = (Probability of Scenario 1 Happening x Scenario 1 Total Costs) + (Probability of Scenario 2 Happening x Scenario 2 Total Costs) + (Probability of Scenario 3 Happening x Scenario 3 Total Costs) + …

  • Expected Total Benefits = (Probability of Scenario 1 Happening x Scenario 1 Total Benefits) + (Probability of Scenario 2 Happening x Scenario 2 Total Benefits) + (Probability of Scenario 3 Happening x Scenario 3 Total Benefits) + …

  • Expected ROI = (Expected Total Benefits - Expected Total Costs) / Expected Total Costs x 100%

Example: Company A calculated the Expected ROI for the project to be 18% as per calculation below:

  • Expected Total Costs: (20% x $148,000) + (60% x $163,000) + (20% x $178,000) = $163,000

  • Expected Total Benefits: (20% x $300,000) + (60% x $180,000) + (20% x $120,000) = $192,000

  • Expected ROI: ($192,000 - $163,000) / $163,000 x 100% = 18%


Step 7: Stack Up Your Alternatives


Once you've detailed the costs and benefits, comparing your project to other investment alternatives (or doing nothing) is essential. The intangible benefits and costs (e.g. time to value) should also be compared alongside the Expected ROI with the other alternatives. By juxtaposing the project's Expected ROI with other options, businesses can make informed decisions about where to allocate resources.


Example: Company A calculated that the Expected ROI for the project is 18% over three years. Comparatively, investing in a new marketing campaign might yield a 10% ROI. In such a case, the project seems to be a better option.


Step 8: Keep Tabs on Outcomes


To ensure the accuracy of your ROI estimates, it is crucial to monitor the progress and outcomes of your project. Regularly review and update the key metrics identified in Step 2 to track the actual performance of your project. This will allow you to make any necessary adjustments and evaluate the effectiveness of your investment. Additionally, consider conducting periodic assessments to identify any potential risks or opportunities that may arise during the project's lifecycle.


With the above, you now can estimate the ROI of your AI/ML or Gen AI project in a systematic approach. This is a very crucial step in the implementation process which will determine the success or failure of the project. What other challenges do you face when estimating the ROI of your AI/ML project? Let me know in the comments below.

246 views0 comments

Recent Posts

See All

Comments


bottom of page