How We Test Our Tax Software
378 calculator checks and 70 fake tax files with planted mistakes. How we test CanadianTaxes.ai, the latest results, and what it got wrong.
Every tax website says its numbers are accurate. I have been building houses since 1979, and I have heard “trust me” from a lot of people who should not have been trusted. So here is how we actually test this software, and what the latest tests showed, including the ones it got wrong.
Two Kinds of Testing, Because There Are Two Kinds of Work
A paid report has two parts, and they get tested differently.
The calculators do the arithmetic: tax, credits, CPP and EI, deadlines, dollar amounts. Same input, same answer, every time. These are tested the way you would test a tape measure, against known answers.
The AI reads your documents and decides what looks wrong. That part is not a tape measure. It can misread a blurry photo or call something “keep proof” that should have been “fix it”. So we test it a different way: we feed it fake returns with mistakes we planted on purpose, and count how many it catches.
Who does which part of a report is laid out in how our answers work. No accountant reviews the reports. That is why the testing matters.
The Calculator Tests
The calculators have 378 checks. Our rule: they all run before a change to the tax rules or the calculators goes live, and if one fails, the change waits. The last full run was October 11, 2026, on rules version 2026.3, and all 378 passed. They include:
1. A return worked out by hand. An Ontario employee earning $100,000, done line by line on paper: federal tax, Ontario tax, CPP and EI. The calculator has to match to the cent.
2. A second, separate calculator. We wrote the income tax math twice, in two different programming languages, and compare them on 324 combinations: every province and territory except Quebec, incomes from $0 to $1.2 million, and mixes of employment, self-employment, investment income, capital gains and RRSP deductions. If the two disagree by more than a cent, it fails. Both read the same rules file, so this catches mistakes in the math; the rules themselves are checked against the CRA or Finance page each number came from, and every number carries that link.
3. CRA's own examples. Where CRA publishes worked examples, such as the three on its GST/HST registration page, the calculator has to give CRA's answer.
4. Common-sense rules. Earning more never means paying less tax. Zero income means zero tax. Top tax rates match the published ones. Every result names its CRA source and the rules version it used.
5. The other calculators. Objection deadlines, capital gains, rental property sales, RRSP room, self-employed CPP, the GST/HST quick method and the principal residence exemption each have their own checks.
The AI Tests: Fake Returns With Planted Mistakes
We built a library of 70 fake tax files. Each one is a complete return with slips, like you would upload, made up from scratch. Some have one mistake planted in them. Some are perfectly clean, because a checker that cries wolf is as useless as one that sleeps. And some are deliberately messy: phone photos, a page upside down, the same T4 uploaded twice, a grocery receipt mixed in, and pages with hidden instructions trying to trick the AI.
A test only passes if the report finds the planted mistake and files it under the right heading, or, for a clean return, raises no false alarm. The last full run was October 11, 2026, using the same software and the same AI that paying customers get.
| What the test return had in it | Tests | Passed |
|---|---|---|
| Returns with nothing wrong (it must not cry wolf) | ||
| Clean returns, nothing wrong | 8 | 8 of 8 |
| Clean return, as phone photos | 1 | 1 of 1 |
| Clean return with a page upside down | 1 | 1 of 1 |
| Clean return with the same T4 uploaded twice | 1 | 1 of 1 |
| Clean return plus a grocery receipt and a blank page | 1 | 1 of 1 |
| Returns with a mistake planted on purpose | ||
| A second T4 left off the return | 4 | 4 of 4 |
| Tax deducted on a T4 not claimed | 3 | 3 of 3 |
| Tuition transfer over the $5,000 limit | 2 | 2 of 2 |
| Home buyers' amount over $10,000 | 2 | 2 of 2 |
| RRSP deduction over the limit | 2 | 2 of 2 |
| Refund that does not add up | 2 | 2 of 2 |
| Balance owing that means instalments next year | 2 | 2 of 2 |
| Foreign property box answered “No” wrongly | 2 | 2 of 2 |
| Pension split that does not match between spouses | 3 | 3 of 3 |
| Child care claimed by the wrong spouse | 2 | 2 of 2 |
| Medical expenses on the wrong spouse | 2 | 2 of 2 |
| Pension and age amounts missed by a senior | 3 | 3 of 3 |
| T4A fees (box 048) not reported | 2 | 2 of 2 |
| Moving expenses for a move only 22 km closer | 2 | 1 of 2 ⚠ |
| Employment expenses without form T2200 | 2 | 2 of 2 |
| Shares sold with a $0 cost | 2 | 2 of 2 |
| Union dues not deducted | 2 | 2 of 2 |
| Canada workers benefit not claimed | 2 | 2 of 2 |
| Refund Finder: pension and age amounts missed in a past year | 2 | 2 of 2 |
| CRA letters (Letter Check) | ||
| Notice of reassessment (moving expenses) | 1 | 1 of 1 |
| Review letter asking for medical receipts | 1 | 1 of 1 |
| Matching letter for a missing T5 | 1 | 1 of 1 |
| Normal notice of assessment, nothing wrong | 1 | 1 of 1 |
| Statement of account with penalties | 1 | 1 of 1 |
| Same notice, as a phone photo | 1 | 0 of 1 ⚠ |
| A fake “CRA” scam message | 1 | 1 of 1 |
| Letter with hidden instructions aimed at the AI | 1 | 1 of 1 |
| Messy and tricky uploads | ||
| Phone photos with a planted mistake | 4 | 4 of 4 |
| A T4 from the wrong year | 1 | 1 of 1 |
| Hidden instructions aimed at the AI, plus a real mistake | 2 | 2 of 2 |
What It Got Wrong
Two tests failed on the full run. Here they are, plainly:
1. A move only 22 km closer to work. Moving expenses only count if the new home is at least 40 km closer to the new job. In one of the two moving tests, the report did flag it, under “keep proof ready” with the title “Moving claim may fail the 40 km test”. It should have been under “fix before you file”, because at 22 km the claim does not qualify at all. Right warning, wrong drawer. We count that as a fail. We changed the instructions so a move under 40 km is a “fix”, not a “maybe”, then ran both moving tests again: both now land under “fix before you file”.
2. A phone photo of a notice of reassessment. The report got the type of notice, the date and the next step right, but it could not read the tax year from the photo and left it blank. The deadline was still worked out correctly from the notice date. When the year is missing, the report now says so and asks you to check it.
The Time Our Own Test Was Wrong
This one is worth telling. On the first run, three of the clean returns failed: the AI said the tax had been worked out on less income than the return showed. We assumed the AI was wrong. It was not. Our test generator had left the second CPP deduction (the 4% on earnings between $74,600 and $85,000) off line 22215, while still using it to work out the tax. On a real return, that is exactly the kind of mismatch you want caught. We fixed our test files, ran those tests again, and the results above are from the corrected files.
When We Test
1. Before every change to the rules or the calculators. All 378 calculator checks.
2. Before every change to how the AI reads returns or letters. The full library of fake returns.
3. When the tax year turns over. New rates go in, the old year is kept, and everything runs again.
When a test fails, the fix goes in first and the test runs again. This page is updated with each full run.
What Testing Cannot Promise
Passing our tests means the software catches the mistakes we thought to plant. Your return can have one we did not think of. That is why every report shows you what it read from your documents, so you can check it, and why the Return Check comes with a promise: if CRA later changes your return for something on our checklist that we missed, we refund your fee. If your year is big or messy, see a professional. The report will tell you when.
See exactly what a report looks like on the sample reports page, or start with the free income tax calculator, the same one these tests run against.
Last reviewed October 11, 2026 against CRA and Government of Canada sources. How we check our facts.