Can AI do investment research?

Commercial AI tools, because they lack access to fundamental or primary data, are unable to answer basic research questions that require pulling and comparing information across many securities at once. They are well suited to working on a single security in isolation — reading an annual report, summarizing a fund's stated strategy, or working from a specific filing that functions as a primary source in its own right. They are not equipped to pull verified, current performance data for hundreds of mutual funds, ETFs, or equities and rank them against one another.

The reason isn't carelessness. It's how these platforms are built. An AI engine doesn't query a database and sort the results — it generates a response by pattern-matching across the vast body of financial text it was trained on, producing whatever fund names, tickers, and return figures are statistically likely to sound correct based on what it has seen discussed most often. That output is then formatted exactly like a real answer: a clean table, specific percentages, and often a cited source. Nothing about the presentation signals whether a figure was retrieved from a live source or reconstructed from memory — the two look identical on the page. Because the model is trained to produce a complete, confident-sounding response rather than flag what it doesn't know, it ends up naming the securities most discussed in its training data — not the best performers — and attaching real-looking numbers to them with no indication that the underlying process was recall, not retrieval.

The design goal was replicability: any advisor with access to the same six platforms should be able to run this test themselves and land on comparable results. That meant fixing every variable we could control — which platforms, which account tier, which prompts, in which order, over how many sessions — so that whatever differences showed up in the output were attributable to the engines, not to how the questions were asked. The exact platforms, versions, and session parameters used are reproduced in full under Replication Parameters in the Appendix.

The study measured three things against each engine: consistency (did it give the same answer to the same question asked three separate times), accuracy (did its answer match verified, current market data), and transparency (would it disclose, when asked directly, where its numbers actually came from and how confident it was in them). A platform could fail on any one of these independently of the others — an engine could be consistently wrong, or accurately right by chance in one trial and wrong in the next, or simply refuse to answer the transparency questions honestly.

Six commonly available commercial AI platforms were tested, each on its paid or pro consumer tier, using the standard consumer chat interface rather than a developer API, an agent mode, or a custom GPT/persona. This was a deliberate choice: it reflects how a financial advisor would actually use these tools day to day. The exact model versions, retrieval tooling, and knowledge cutoffs used for each — ChatGPT, Claude, Copilot, Gemini, Grok, and Perplexity — are listed in the Appendix.

Five prompts were used, and the same five were used for every engine, worded identically every time. Prompts 1, 2, and 3 are the core performance questions — one per asset class — and each was run as the opening prompt of its own session. Prompts 4 and 5 are diagnostic: they don't ask for new performance data, they interrogate the answer the engine just gave, immediately afterward, in the same session. Each engine ran each asset-class chain three times, in a fresh private/incognito session every time, so nothing carried over from memory between trials.

Asked each engine for the top five Canadian equities by trailing 12-month return, filtered to Canadian-domiciled issuers with market cap over $1 billion CAD, with six supporting data points per security — market cap, share price, P/E, dividend yield, 3-year price appreciation, and beta.

Asked each engine for the top five Canadian equity or Canadian-focused equity mutual funds by 3-year annualized return, with nine supporting data points per fund — FundServ code, MER, 3- and 5-year return, standard deviation, drawdown, AUM, and CIFSC category.

Asked each engine for the top five Canadian-listed ETFs by 3-year annualized return, with seven supporting data points per ETF — ticker, issuer, MER, AUM, 3-year and 1-year return, and average daily volume.

Run immediately after every core performance prompt, this asked the engine to account for its own answer: enumerate every source it relied on, classify each as primary or context-only, disclose whether each was actually fetched during the session or only recalled, rate its own confidence per data point, and state directly whether it had reached Bloomberg, FactSet, Morningstar Direct, S&P Capital IQ, or SEDAR+. This is the prompt behind the sourcing findings discussed in Citations & Sourcing below.

Closed every session by asking the engine to document its own technical conditions — model version, knowledge cutoff, whether live retrieval was active, and similar metadata — so a later replication attempt has something concrete to compare its own session against.

The verbatim wording of all five prompts, the complete list of required data points and disclosure fields for each, and the conditions under which they were run are reproduced exactly under Replication Parameters in the Appendix.

Every individual trial ran in its own private, incognito browser session, opened immediately before the trial began and closed immediately after the response was captured. An AI engine can carry context from one query to the next within an open session, so starting fresh for every single trial — not just every asset class, not just every engine — was the only way to guarantee that no trial was quietly informed by anything that came before it.

Within a given engine, the three asset classes were run back-to-back: one asset class was taken through all three of its trials before moving to the next, each trial using the same three-prompt chain for its asset class (the core performance prompt, followed by the Prompt 4 sourcing audit, followed by the Prompt 5 session-metadata capture). The same sequence was then repeated, asset class by asset class, for each of the six engines, producing 54 trials in total. A worked example of the full nine-trial sequence for one engine, and the complete execution count across all six, are reproduced under Replication Parameters in the Appendix.

Once all 54 responses were captured, each was coded along the same three dimensions the study was designed to measure — consistency, accuracy, and transparency, defined precisely under the Appendix's Scoring Method. That coding is what produced the dispersion map in the Findings section below, and the verified-versus-returned comparisons in the Accuracy section that follows it.

Consistency was tested by asking each engine the identical question three separate times, in three fresh sessions, and comparing what came back. Across 54 trials the results split into two failure patterns, and neither one is reassuring. Engines were either locked onto the same wrong answer every time, or they were volatile — swinging between unrelated answers to the exact same prompt, with no warning in the output that anything had changed.

Where engines were consistent, they were consistently wrong. Grok produced the same five ETFs in identical order across all three ETF trials — the highest ETF consistency of any engine in the study, and the audit prompt confirmed it reflected a memorised list rather than any live screening. Perplexity's ETF Trials 2 and 3 were byte-identical, consistent with the same pattern: a stable training-data snapshot standing in for a real answer. Claude's top four ETF picks held constant across all three ETF trials, making it the most internally stable engine on that asset class — and Claude's third mutual-fund trial was refused outright, a refusal that reproduced identically on a rerun. Even a refusal, in other words, can be a "consistent" result. An engine that gives you the same wrong answer, or the same non-answer, three times in a row is not consistent in any meaningful sense. It is stuck.

The more common and more troubling pattern was significant variation between trials of the identical prompt. Gemini posted the worst record in the study on both fronts at once: zero live retrieval in all nine of its trials, fabricated as-of dates on every figure, and the lowest cross-trial consistency of any engine — it did not even reliably repeat its own fabrications. Copilot's equity Trials 1 and 2 were byte-identical before Trial 3 returned a completely unrelated list of micro-cap stocks with no overlap from the previous two — and in a separate Equity Trial 2 run, Copilot's own Prompt 4 disclosure described its own numbers as "FABRICATED PLACEHOLDERS, not retrieved from any source," with nothing in the original table hinting at that. Perplexity refused two of its three equity trials outright and answered the third, meaning the same prompt, asked three times, produced a coin-flip between a refusal and a full formatted answer.

Celestica appeared in 14 of 18 equity trials — the single most repeated pick across every engine and every trial. Its actual one-year return placed it outside the verified top five by more than 300 percentage points. That level of consensus tells us nothing about performance. It tells us which company generated the most financial media coverage, analyst attention, and training-data presence. Agreement between engines, like agreement between an engine's own trials, turned out to be a measure of shared exposure to the same training and search content — not a measure of being right. The full citation count behind every figure in this section — every security, every engine, every trial — is reproduced in the Appendix.

Accuracy was checked the only way that means anything: against Buckler's verified market data for the same period, as of April 30, 2026. Across all three asset classes and all 54 trials, the gap between what the engines returned and what the verified data showed was not marginal. It was categorical. The engines were not slightly off on the numbers for the right securities. They were identifying a different set of securities entirely — ones that happened to be well-known, widely covered, and prominent in financial media, regardless of their actual performance by the metric each prompt specified.

Every "Citation Rate" figure below is the share of that asset class's 18 trials (six engines, three trials each) in which the engine's answer included that specific security.

The verified top five Canadian equities by one-year trailing return were Almonty Industries (+685.9%), Groupe Dynamite (+587.5%), Hut 8 Mining (+506.8%), Faraday Copper (+460.5%), and Spartan Delta (+417.3%). Across all 18 equity trials — six engines, three trials each — four of those five names were never cited once. Faraday Copper was cited exactly once, a 5.6% citation rate. Almonty Industries, the single best-performing Canadian equity in the entire dataset, up nearly 686% over the period, was named by zero engines in zero trials: a 0% citation rate.

Celestica was the engines' consensus pick — a 77.8% citation rate, more than any other name by a wide margin — and its actual return placed it more than 310 percentage points below the verified number one. That is the clearest illustration in the study of what "consensus" from these engines actually measures: not performance, but how often a name shows up in the training data and search results the engines draw on. Celestica is a large, widely covered technology manufacturer. Almonty Industries, a smaller mining company, is not — and that difference in media visibility, not in return, is what determined which one the engines converged on.

The verified top five Canadian equity mutual funds by three-year annualized return were Purpose Global Resource Fund Series L (+52.3%), Friedberg Global-Macro Hedge Fund U$ (+50.1%), CI Precious Metals Fund Series I (+50.1%), Dynamic Precious Metals Fund Series O (+48.3%), and Ninepoint Silver Equities Fund Series D (+47.3%). None of the five appeared in a single trial, by any engine, at any point in the study — a 0% citation rate across all 18 mutual fund trials.

The fund the engines cited most often, Guardian Canadian Focused Equity Series F, still only reached a 16.7% citation rate — under the 20% line — and returned 23.8% over three years, less than half of the actual top performer. And the miss went beyond picking the wrong fund: four of the 18 mutual fund trials didn't return mutual funds at all. The engines substituted ETFs instead, meaning in more than one in five trials, the response failed to match even the asset class the prompt specified, before any question of which specific fund was correct.

The verified top five Canadian-listed ETFs by three-year annualized return were concentrated in gold and precious metals: BMO Equal Weight Global Gold Index ETF and iShares S&P/TSX Global Gold Index ETF (tied at +53.7%), BetaPro Gold Bullion 2x Daily Bull ETF (+50.0%), BMO Junior Gold Index ETF (+48.7%), and Global X Global Semiconductor Index ETF (+47.5%). Unlike equities and mutual funds, the engines did land in the right neighbourhood here — gold ETFs were a recurring theme across trials — but never on the specific, correctly ranked winners, and the study's single most-cited ETF overall did not even meet the prompt's filter criteria.

The engines' gold ETF appearances reflect training-data familiarity with a well-known category of fund, not a live performance screen that correctly ranked one gold ETF over another — none of the five verified winners cleared a 40% citation rate, and the semiconductor ETF that placed fifth sat at just 5.6%. More striking is what the engines cited most often overall: CI Galaxy Bitcoin ETF, at a 55.6% citation rate, ahead of every verified top-five gold ETF. It does not meet the Canadian-equity filter specified in the prompt, and its 36.7% three-year return sits nearly 17 points below the actual top performer. The engines' most confident, most frequent answer was not just short of the correct one — it was disqualified by the prompt's own criteria.

Every citation count and verified return referenced in this section — including every security cited but not shown above — is reproduced in full in the Appendix, as a standing reference for checking any accuracy claim in this document against the underlying data.

Prompt 4, run immediately after every core performance question, asked each engine to account for itself directly: where did you source this information? Where did most of the sourcing actually come from? How did you obtain the data? And, specifically, where did you actually get it, as opposed to where you said you got it? The answers, run across all 54 trials, establish something the original responses never disclosed on their own.

None of the six engines accessed any institutional financial database in any trial. Bloomberg, FactSet, Morningstar Direct, S&P Capital IQ, SEDAR+, and exchange-level data feeds — the tools that would actually support a ranked, verifiable screen across hundreds of securities — were never reached, not once, by any engine. Several engines named these platforms in their output anyway. What the engines used in practice was a mix of training data, web search snippets, and in several cases, nothing at all.

An AI engine is not a search engine and it is not a database. It is a language model trained to predict what a helpful, coherent response looks like. When it cannot retrieve the data a question requires, it does not stop. It produces what a correct answer would look like based on patterns in its training data — including plausible figures, plausible fund names, and plausible citations. Every engine in this study has a knowledge cutoff of between late 2023 and mid-2024, leaving a gap of twelve to twenty months between its last training snapshot and the May 2026 prompt. A response that includes a formatted table, specific return figures, named sources, and an as-of date looks like a researched answer. There is no way to tell from the output alone that it is not — the only way to know is to ask, and the engines were asked in every one of the 54 trials.

The clearest illustration of the gap between a claimed source and an actual one came from Perplexity, the engine that cited more paywalled institutional sources than any other in the study — ten mentions across all trials — without retrieving a single one of them. In one mutual fund trial, the engine's own output referenced Morningstar Direct, an institutional research terminal that costs thousands of dollars per seat annually and implies direct, licensed data access. When asked under Prompt 4 to confirm whether it had actually reached Morningstar's terminal, the audit told a different story:

The distinction is not a technicality. One is a data retrieval, priced and licensed specifically because it is authoritative. The other is a name appearing in the first sentence of a public webpage, reproduced as if it were a source attribution. A reader relying on the original response — the one without the audit prompt attached — has no way to tell the two apart, because the engine formatted its citation of the article description identically to how it would have formatted a genuine terminal pull.

Grok went a step further: in two of its nine trials, it produced source citations that were formatted correctly, named plausible-sounding institutions, and appeared directly alongside figures presented as retrieved data. There was nothing about the citations themselves that would flag them as suspect. When audited, Grok acknowledged the figures had not come from any specific source at all:

A citation that names a real institution but was never actually consulted, and a citation that was fabricated outright and attached to an approximated figure, produce the same output on the page: a footnote that looks sourced. The only way to tell the difference between a genuine citation, a misattributed one, and an invented one is to ask the engine to account for it directly — and every engine in this study required that follow-up before it would say so.

Run across all six engines, Prompt 4's four questions — where the information was sourced, where most of the sourcing actually came from, how the data was obtained, and where it actually came from versus where the engine said it came from — produced a consistent pattern: what the original response implied and what the audit revealed were rarely the same thing.

The complete citation record behind these findings — every security any engine named, how many times, and against the verified return — is reproduced in full in the Appendix, as a standing reference for checking any consistency, accuracy, or sourcing claim made in this document against the underlying data.

Across 54 trials, three failure patterns kept repeating regardless of engine, asset class, or trial number: no engine consistently reproduced its own answer, no engine returned a verified top performer in any category, and no engine disclosed the limits of its data unless asked directly. That isn't a list of bugs to patch. It's a structural limitation in what commercial AI can currently do with a research question that requires pulling and ranking real data across many securities at once. Four conclusions follow.

A neatly formatted top-five table with tickers, exact percentages, citations, and an as-of date looks identical whether every figure was retrieved from a live source or invented outright. The presence of a citation is not evidence that it was fetched — the Prompt 4 sourcing audit caught engines admitting, after the fact, that specific numbers were "simulated," "approximated," or "FABRICATED PLACEHOLDERS, not retrieved from any source." Nothing in the original response distinguished those figures from the ones that were real. An advisor reading the output has no way to tell which is which without a separate verification step the AI will not prompt them to take.

Where engines agreed — with each other, or with their own earlier trials — that agreement tracked media coverage and training-data presence, not returns. Celestica appeared in 14 of 18 equity trials, more than any other security in the study, and its actual one-year return placed it outside the verified top five by more than 300 percentage points. Repeating a question, or checking a second engine's answer against the first, does not independently confirm anything if both are drawing on the same underlying exposure. Cross-checking AI engines against each other is not verification.

This study used long, explicit, tightly filtered prompts — market-cap thresholds, CIFSC categories, exact ranking criteria, mandatory data points, "N/A" instead of a decline to answer — and it made no measurable difference. No amount of prompt engineering closes this gap, because the limitation isn't how the question is asked. It's that none of the six engines have access to institutional-grade primary data for Canadian securities: no Bloomberg terminal, no FactSet feed, no Morningstar Direct workstation, no FundServ record.

Closing that gap isn't simply a matter of an AI lab deciding to add it. Bloomberg, Morningstar, FactSet, and Thomson Reuters built their businesses on licensing that data at a metered, per-seat cost to institutions and professionals — often tens of thousands of dollars a year, priced around scarcity and exclusivity. Opening that data to a consumer AI platform would mean putting it, in practice, in front of every user of that platform — hundreds of millions of people — at effectively no marginal cost. That isn't a pricing adjustment for a data provider; it's the collapse of the business model the data was built to fund. There is no obvious commercial path by which Bloomberg-grade data becomes ChatGPT-grade data, because doing so would undercut the reason Bloomberg-grade data is valuable in the first place. That's a structural reason to expect this gap to persist even as the underlying models keep improving.

None of this is an argument against using AI. These engines are genuinely useful for orientation, idea generation, drafting, summarising, and brainstorming. They are not reliable for the kind of factual retrieval an advisor or investor needs without an explicit verification step — and the responses give no indication, on their own, that such a step is necessary. An advisor relying on this output risks presenting clients with securities selected by coverage rather than by returns, formatted with exactly the same confidence as if the numbers were real.

The root cause is simpler than the failure modes make it sound: AI engines do not have access to high-quality primary data for Canadian securities, so when asked for a current return, MER, or AUM, they substitute whatever they can reach — rarely the right source, never a complete dataset. Better prompts will not fix a missing dataset. Producing replicable, consistent, and accurate results takes four things working together, not just a better-trained model.

Not web snippets, not training-data approximations, not a rate-limited free API tier — the same class of structured, current, institutional-grade data that Bloomberg, FactSet, Morningstar Direct, S&P Capital IQ, and SEDAR+ already provide to professionals. That access has to be licensed and paid for like any other institutional data feed; there is no free version of this that holds up.

The model itself should never be the source of a number. The figures a user sees need to come from a live query against a structured database — ranked and filtered deterministically — with the language model used only to interpret the question and format the answer, not to generate the data inside it. That is the difference between retrieval and recall, and it is the difference this entire study measured.

Every number displayed needs a traceable source, a retrieval timestamp, and a real as-of date attached automatically — not produced only when an engine is directly asked to disclose its sourcing, as the Prompt 4 audit had to force here. If a data point cannot be sourced, the honest answer is "not available," not a plausible-looking placeholder.

The same question, asked twice, needs to return the same answer, because it is drawing from the same underlying dataset rather than regenerating a plausible-sounding response from memory each time. Consistency in this study was mostly coincidence — engines that agreed with themselves were often just replaying the same memorised list. Real consistency comes from a shared source of truth, not a shared training set.

The tables below list every security named by any engine, in any trial, across the study — the underlying ledger that the Consistency, Accuracy, and Citations & Sourcing findings are drawn from. Times Cited counts appearances across all 18 trials for that asset class (six engines, three trials each). Bolded, highlighted rows are the actual top five by verified return; as the Findings section shows, none were identified by any engine.

The tables below show each engine's top-five picks (or fewer, where an engine refused or ran out of answers) in each of its three trials, for each asset class. Colour marks how often a security recurred across that engine's own three trials — the same measure used throughout the Consistency findings above.

Celestica's 14-of-18 citation rate is visible here at the individual-trial level: every engine except Perplexity named it, and ChatGPT and Grok named it in all three of their own trials. Grok's identical Trial 1 and Trial 2 picks, followed by a two-name Trial 3, and Copilot's byte-identical Trials 1–2 followed by an entirely unrelated Trial 3, are the same events described narratively in the Consistency section above — this is the underlying data they were drawn from.

Every mutual fund trial returned different names, engine to engine and often trial to trial. Some engines substituted ETFs outright; the source data also shows the Guardian Canadian Focused Equity Fund appearing under more than one series code within a single engine's own trials, reflecting ambiguity in how the engines resolved FundServ codes rather than a single consistent pick.

Grok's ETF picks are identical across all three trials — the exact pattern described in the Consistency findings as reflecting a memorised list rather than a live screen.

The parameters below are the complete set of conditions used to produce the 54 trials in this study. Holding all of them constant — same prompts, same order, same session discipline — is what makes a second run of this test comparable to the first one, even months later on different model versions.

• Period: May 8–9, 2026 — Mississauga, Ontario, Canada.
• Sessions: One fresh, private/incognito browser session per trial, opened immediately before the trial and closed immediately after the response was captured. No session was reused across trials, asset classes, or engines.
• Trial count: Three independent trials per engine per asset class (18 trials per asset class, 54 trials total across all six engines).
• Trial order: Within a given engine, one asset class was taken through all three of its trials before moving to the next — not interleaved. The exact sequence is worked through below.
• Mode: Default consumer chat mode on each platform's standard web app. No deep research, extended thinking/reasoning, agent, or custom GPT/persona modes, and no developer API.
• Account tier: Paid/pro consumer tier on every platform, to reflect how a financial advisor would actually use these tools.
• Prompt wording: Identical wording, character for character, across every engine and every trial — see the five prompts below.
• Verification baseline: Every returned security and figure checked against Buckler's verified market data as of April 30, 2026.

Every engine followed this same pattern: one asset class taken through all three of its trials, in a fresh incognito session each time, before moving to the next asset class.

At each step, the engine's full response to the core performance prompt was captured verbatim, followed immediately by its answers to the Prompt 4 sourcing audit and the Prompt 5 metadata request, all inside the same session. Nothing was summarized, cleaned up, or paraphrased at capture time. The same nine-step sequence was repeated for each of the remaining five engines.

Each session ran one asset-class chain, opening prompt first: Equities = Prompt 1 → Prompt 4 → Prompt 5. Mutual Funds = Prompt 2 → Prompt 4 → Prompt 5. ETFs = Prompt 3 → Prompt 4 → Prompt 5. No follow-ups, rewording, or clarification requests were used at any point.

Filters: issuer domiciled in Canada; market capitalization greater than $1 billion CAD, ranked by 12-month trailing total return. Required six data points per security (market cap, share price, trailing P/E, trailing 12-month dividend yield, 3-year price appreciation, 3-year beta vs. S&P/TSX Composite), single table, ticker/issuer/exchange included, as-of date stated, no unsolicited commentary.

Filter: CIFSC category of Canadian Equity or Canadian Focused Equity, ranked by 3-year annualized return. Required nine data points per fund (fund name, FundServ code, MER, 3-year and 5-year annualized return, 5-year standard deviation, maximum drawdown, AUM, CIFSC category), N/A for unavailable data rather than declining to answer.

Filter: listed on a Canadian exchange. Required seven data points per ETF (ticker, issuer, MER, AUM, 3-year annualized return, 1-year return, average daily volume), same table and N/A conventions as Prompt 2.

Run immediately after the core prompt in every session. Required the engine to classify each source as primary (issuer fact sheets, KYP documents, Bloomberg, FactSet, Morningstar Direct, S&P Capital IQ, exchange filings, SEDAR+) or context-only (guidance articles, blogs, Wikipedia); to state whether each URL was actually fetched, used only as a search-snippet reference, a placeholder, or never retrieved at all; to give a confidence rating per data point (HIGH/MEDIUM/LOW); to confirm directly whether Bloomberg, FactSet, Morningstar Direct, S&P Capital IQ, issuer KYP/fact-sheet PDFs, SEDAR+, and exchange-level feeds were accessed; and to disclose retrieval failures. Closed with: "If you cannot answer any portion of this disclosure honestly, say so directly rather than inferring."

Ten items requested per session: date/time and timezone as reported; model name and exact version; knowledge cutoff; whether live browsing/search was enabled and which retrieval engine; whether any deep-research, reasoning, or extended-context mode was active; custom instructions, system prompt, persona, or memory settings in effect; inferred or stated geographic/regional session settings; a session/conversation/response ID if available; tool restrictions, rate limits, or capability constraints; any other metadata a replicator would need. Closed with: "If you cannot answer any of the above, state so explicitly rather than guessing."

• Consistency: an engine's three trials for a given asset class compared against one another — same securities, same order, same figures, or not.
• Accuracy: securities and return figures compared against Buckler's verified market data as of the trial period; citation rate calculated as the count of trials (of 18 per asset class) in which a security was named, expressed as a percentage.
• Transparency: the engine's own Prompt 4 and Prompt 5 answers checked against what it had actually done — whether a claimed source was real and fetched, a snippet reference, a placeholder, or fabricated outright.