January 15, 2025
Assessing AI From a Tax Perspective, Part 1
Evaluating AI's potential and limitations to assist with tax-related queries during tax season
By William M. VanDenburgh, Ph.D.; Kimberly Jane Tribou, Ph.D., CPA; and James M. Braswell, Ph.D.
The allure of artificial intelligence (AI) was highlighted daily when ChatGPT became publicly available back in late 2022. A February 2024 Wall Street Journal (WSJ) front-page headline boldly declared, “AI Is Starting to Threaten White-Collar Jobs.”1 Citing an August 2023 working paper published by the University of Pennsylvania and OpenAI, the WSJ contended that “accountants are among the professionals whose careers are most exposed to the capabilities of generative artificial intelligence.”2
Ideally, generative AI could make the annual tax season process more efficient and effective; this could help with the shortage of accountants.
In this article, we ask Copilot (a commonly available AI tool) questions that arose during the 2023 tax filing season to assess the quality of answers provided. We had high hopes that Copilot would aid in the tax season push, but we found tax AI answers were often wrong and/or misleading. A companion article in the next issue of Today’s CPA (“Assessing AI From a Tax Perspective, Part 2”) will compare three commercially available AI tools, offer preparer pointers and identify risks in using AI.
Examining AI
We evaluated the usefulness of AI in researching tax issues that arose while preparing client tax returns during the 2023 tax filing season (February to April 2024). While tax professionals may well find AI useful at times, the way questions are asked is crucial to the results.
Additionally, tax practitioners must exercise extreme care in reviewing AI’s answers, which may be incomplete and incorrect (in AI terms, a “hallucination”). When asking AI tools to research complicated tax areas, such as state and local taxation and trust taxation, AI tended to provide hedged responses. For example, Copilot tended to respond to tax questions with this or a similar warning:
Please note that tax matters can be complex and individual circumstances may vary. It’s advisable to seek professional advice for accurate and personalized guidance.
How questions are phrased significantly affects the accuracy of AI's responses, highlighting a need for users to be precise and critical when querying AI.
Overall Assessment
We primarily utilized Copilot (Microsoft AI), which offered several advantages for our study. Copilot incorporates the newest generative pretrained transformer (GPT) technology offered by OpenAI, the company that licensed ChatGPT. Copilot has been integrated within Microsoft Windows 11 since November 2023, making it easily accessible to tax professionals and the public. Finally, Copilot offers three different answer modes (i.e., more creative, more balanced and more precise), allowing the user to select the precision level that best complements their expertise.
While tax professionals may be inclined to use the “More Precise” mode, posing the same question in the “More Creative” or “More Balanced” modes allows preparers a glimpse into the tool’s reasoning, which may be helpful when studying tax questions and verifying the technology’s accuracy.
When inquiring about a general tax issue (i.e., the teacher education expense of $300 in 2023), Copilot could draft a client letter that was responsive to our prompt. However, overall, it was a D+ student if the professor was kind.
At times, the tool provided different answers depending on which mode was selected. For example, when asked to calculate a reasonable estimate for tax basis of a stock, Copilot could not provide the correct determination and provided different determinations under each answer mode. Further, AI gave conflicting answers as to the total amount of tax due on a Fiduciary state tax return. Table 1 summarizes Copilot’s erroneous and/or misleading results.
| Table 1. Key Responses to Copilot Prompts | ||
| Question | Copilot Response | Our Response Assessment |
| Closing value of Coca-Cola stock on October 5, 1995 | Copilot stated the “closing price for Coca-Cola (KO) on October 5, 1995 is not directly available. However, the closing price for Coca-Cola (KO) in 1995 was $9.30 on December 29, 1995.” It recommended checking a “detailed historical stock price database.” | Overall poor: The split-adjusted value of Coca-Cola stock on October 5, 1995 is $17.66. The actual closing value was $70.64, but it needs to be adjusted for two for one splits in 1996 and 2012. |
| If the Louisiana Fiduciary tax return for 2023 has taxable income of $140,525, what is the total tax due?3 | In the “more balanced” mode, it stated the tax “would be approximately $7,231.50.” When re-asked in “more precise” mode, it calculated the correct amount of $5,432.31. | Overall bad and good: This straightforward calculation should be easily determined, regardless of the mode selected. Initially, it was calculated based on rates of 2%, 4% and 6% (the rates in 2021).4 The correct rates for 2023 are 1.85%, 3.5% and 4.25%. |
| SPAXX % of U.S. government securities held in 2023 | “As of 2023, the Fidelity Government Money Market Fund (SPAXX) was reported to have 30.59% of its holdings in U.S. government securities for state tax purposes.” Copilot recommended checking with Fidelity for updated information. | Fair to Poor: The 30.59% figure was for 2022. The 2023 percentage was not available as of the asking date. Copilot did recognize the fund symbol. When Fidelity released the 2023 percentage on February 15, 24, Copilot correctly linked to the document, but did not update its response with the correct percentage (41.18%). |
Stock Tax Basis
When tax clients sell a stock investment acquired before 2011/2012, the determination of the stock basis can be problematic. The Energy Improvement and Extension Act of 2008, P.L. 110-343, mandated that financial firms track the basis of clients’ assets and report it to their clients and the IRS upon sale. The phased implementation of this rule occurred between 2011 and 2013.5
Determining historical stock basis is a task, we thought, that could be safely delegated to AI. Calculating stock basis uses historically available stock data that is readily available on investment websites (e.g., Yahoo! Finance). Table 2 presents the manually calculated stock basis of 1,000 shares of Coca-Cola stock with a valuation date of October 5, 1995.
| Table 2. Determining an Heir’s Cost Basis in Coca-Cola Stock | |
| Stock inherited | Coca-Cola |
| Valuation Date | 10/5/1995 |
| High value 10/5/95 | $71.88 |
| Low value 10/5/95 | $70.25 |
| Average high and low value on 10/5/95 | $71.065 |
| 1996 - 2 for 1 stock split effect | $35.5325 |
| 2012 - 2 for 1 stock split effect | $17.7663 |
| Heir’s cost basis per share as of 2023 | $17.7663 |
| Heir’s basis of 1,000 shares of Coca-Cola | $17,766 |
While Copilot knew the Energy Improvement and Extension Act requirement and its history, it could not determine the tax basis of the stocks we queried (Coca-Cola, ExxonMobil, Bristol-Myers Squibb, and Apple). Where Copilot sometimes gave an estimated value, it was often materially different than the actual shareholder’s basis. Table 3 shows the stock bases we requested, summaries of Copilot’s responses and our critique of its answers.
| Table 3. Stock Basis Copilot Queries | ||
| Question | Copilot Response | Our Response Assessment |
| What is an heir’s basis per share of Coca-Cola (KO) stock inherited on October 5, 1995 for U.S. federal tax purposes? | Copilot’s “Creative Mode” provided an adjusted closing price of $23.25 per share on the reference date. The tax basis for 1,000 shares would thus be $23,250. | Wrong: Per our manual calculation (Table 2), Coca-Cola’s split-adjusted value for stock inherited on October 5, 1995 would be $17.77 and the tax basis of 1,000 shares would be $17,766. |
| What is an heir’s basis per share of ExxonMobil stock inherited on October 5, 1995 for U.S. federal tax purposes? | It gave general advice in all three modes on how to determine a basis that was not technically correct. | Unable to Determine: ExxonMobil’s split-adjusted value for stock inherited on October 5, 1995 would be $18.4063. It did not mention the need to average high and low values for inherited value based on the valuation date, which could be the date of death or the alternative valuation date. |
| What is an heir’s basis per share of Bristol-Myers Squibb Co. stock inherited on October 5, 1995 for U.S. federal tax purposes? | Copilot referenced the FMV on the date of death, but did not give a value (creative mode). | Unable to Determine: Bristol-Myers’ split-adjusted value for stock inherited on October 5, 1995 would be $17.94. The average high and low value was $75.38, and there were 2-for-1 splits in 1997 and 1999. Its basis was further reduced by 4.83% when in 2001, Zimmer Holdings was spun off. |
| What is the split-adjusted cost basis of Apple shares purchased on January 27, 2006? | It gave the steps to calculate the cost basis. It gave an example based on buying 10 shares at $10 each in 2006 and stated, therefore, the basis “is approximately $0.1786 per share.” | Unable to Determine: The split-adjusted closing price of Apple shares purchased on January 27, 2006 is $2.57.6 |
When we asked for the stock basis for an heir, Copilot referenced “the fair market value of the stock on the date of the decedent’s death,” which is ambiguous at best. Per IRS Instructions for Form 706, United States Estate (and Generation-Skipping Transfer) Tax Return, “the FMV of a stock or bond (whether listed or unlisted) is the mean between the highest and lowest selling prices quoted on the valuation date.”7 The valuation can be the date of death or the alternative valuation date (if elected by the executor for the entire estate).
Instead of asking for a calculation of stock basis, we prompted Copilot for the closing value of Coca-Cola on October 5, 1995. Concerningly, Copilot could still not return the correct answer, stating, “the closing price for Coca-Cola (KO) in 1995 was $9.30 on December 29, 1995.” The actual split-adjusted closing value as of December 29, 1995 is $18.56, which can simply be determined with Yahoo Finance's historical data. Apparently, Copilot returned the value of the adjusted close and even this appears slightly off. The adjusted closing value incorporates “splits, dividends, and/or capital gains distributions.”8 Taxable dividends do not reduce one’s tax basis.
To see if Copilot could better handle more recent stock transactions, we later asked Copilot for the price of Apple shares purchased on January 27, 2006. In the “more creative” and “more balanced” modes, Copilot listed the steps to determine the price but could not determine it. Copilot’s response provided a hypothetical example based on if “you bought 10 shares at $10 each in 2006,” your basis per share would be $.1786. If you bought shares of Apple on January 27, 2006, you would have a cost basis per share around $2.58 (based on its closing value).
Tax Professional Tip: For the stock tax bases we inquired about in Table 2, we found that Yahoo Finance had historical data closing prices (which are adjusted for splits) that matched our stock basis records on the per-share amount.9
Education Credits and QTPs
Copilot's responses to prompts on education credits and Qualified Tuition Plans (QTP) were the most accurate tax concerns we examined. That said, Copilot’s seemingly correct answers were incomplete and could easily lead to incorrect tax filings if relied upon by taxpayers or tax professionals. Please see Table 4.
| Table 4. Education Tax Provision Queries | ||
| Question | Copilot Response | Our Response Assessment |
| Per the IRC, does the cost of room and board qualify for education credits? | Does not | Correct |
| Per the IRC, is the cost of room and board an eligible education expense for a QTP distribution? | Yes, “if the QTP plan allows it, room and board expenses qualify as part of the higher education expenses, provided they are within certain limits.” | Partially Correct: It failed to state the cost is limited to the amount charged by the school or the “actual amount charged if the student is residing in housing owned or operated by the school.”10 |
| Per the IRC, does the cost of a computer qualify for education credits? | “According to the IRC, the cost of a computer may qualify for education credits under certain conditions.” When asked in all three modes, none were perfectly correct. | Partially Incorrect: Computers only qualify for the AOTC if required by the education institution. Computer costs do not qualify for LLC. |
| Per the IRC, is the cost of a computer an eligible education expense for a QTP distribution? | Yes, in all modes. However, the precise mode stated, “this information is current as of my last update in 2021, and I recommend consulting a tax professional or the IRS website for the most up-to-date information.” | Correct: But troubling that Copilot stated its last update was in 2021. |
These inaccuracies may arise as qualifying education expenses for American Opportunity Tax Credit (AOTC), Lifetime Learning Credit (LLC) and QTP distributions vary. Determining eligible expenses and their interaction is difficult at best and can even require allocation between education credit and QTP of qualified expenses.
For example, Copilot indicated that the cost of a computer could possibly qualify for both AOTC and LLC. A computer only qualifies for AOTC if required by the university and does not qualify for the LLC. Copilot correctly responded that a computer is an eligible education expense for QTP, but this conclusion was based upon information that was last updated in 2021. Specifically, it stated (emphasis added):
“Yes, according to the Internal Revenue Code (IRC), the cost of a computer is considered a qualified higher education expense for a Qualified Tuition Plan (QTP) distribution. This means that funds from a QTP, also known as a 529 plan, can be used tax-free to purchase a computer for the beneficiary’s use while they are enrolled at an eligible educational institution. Please note that this information is current as of my last update in 2021, and I recommend consulting a tax professional or the IRS website for the most up-to-date information.”
AI is Helpful at Times, but Not Yet Reliable
Using AI to help answer actual tax season questions in the 2023 tax filing season revealed that AI was of limited use and could easily lead to incorrect tax conclusions. Preparing a quality individual tax return is typically a highly complicated process requiring the integration of federal, state and local tax laws, IRS Regulations, and supporting case law.
Partnership and trust returns – and their K-1s – add another layer of complexity. While AI was helpful at times, it is not yet a substitute for the professional judgment of tax professionals. Our forthcoming companion article, “Assessing AI From a Tax Perspective, Part 2,” will examine the historical, professional and ethical considerations for preparers when using these tools.
Related CPE TXCPA offers these webcasts and more on artificial intelligence topics and emerging technologies:
For details and to register, go to the Education area of the TXCPA website and use the search term artificial intelligence. |
About the Authors: William M. VanDenburgh, Ph.D., is an Associate Professor of Accounting at the College of Charleston (vandenburghbm@cofc.edu). Kimberly Jane Tribou, Ph.D., CPA, is an Assistant Professor of Accounting at the College of Charleston (triboukj@cofc.edu). James M. Braswell, Ph.D., is an Associate Professor of Accounting at the College of Charleston (braswelljm@cofc.edu).
Footnotes
1. https://www.wsj.com/lifestyle/careers/ai-is-starting-to-threaten-white-collar-jobs-few-industries-are-immune-9cdbcb90, 2/12/24.
2. https://www.wsj.com/articles/the-jobs-most-exposed-to-chatgpt-e7ceebf0, 3/28/23.
3. https://www.journalofaccountancy.com/issues/2013/sep/20137345.html, 9/1/13.
4. www.irs.gov/pub/irs-pdf/i706.pdf, page 25.
5. https://finance.yahoo.com/quote/KO/history?period1=819590400&period2=820627200, 4/27/24.
7. Actually asked initially, “If the Louisiana’s Fiduciary tax return for 2023 has taxable income $140,525 what is the total tax due?” Questions we asked in the tables at times varied slightly, as this one did.
8. Louisiana Department of Revenue, "General Information and Instructions for Completing Form IT-541 Fiduciary Income Tax.”
9. www.irs.gov/pub/irs-pdf/p970.pdf, page 51.
Thanks to the Sponsors of Today's CPA Magazine
This content was made possible by the sponsors of this issue of Today's CPA Magazine:
