Improve MCP answers and data quality ahead of the public launch

[JerBouma]

Fixes from a full test of the MCP server before its public launch: more than 80 tool calls on a local server running main, re-run after each round of fixes on the corrected data.

MCP server and package

• Primary listings.
  • A venue's size now counts only the listings of its own country, so a venue full of foreign lines (Hamburg, Xetra) no longer wins as "the largest main exchange abroad". Airbus is AIR.PA, Eurofins ERF.PA, Yangzijiang BS6.SI.
  • A line abroad whose symbol shape almost only receipts have is a receipt, so B3's BDRs/DRNs ('34', '32') are never primary (TMCO34.SA, JBSS32.SA, PHGN34.SA, BOXP34.SA).
  • Abroad, a line with the share's ISIN beats a line without one (Glencore stays GLEN.L instead of GLC.AX).
  • Fixed a crash in the receipt check when every line abroad has a one-word name.
• Retired classes. Asking for funds or money markets now says they are no longer maintained and that the Python package still has them, instead of a raw validation error.
• Currency names. A currency's name finds its pairs ('japanese yen' → the JPY pairs); the names come from the pairs' own descriptions.

Data

• Equity classification. The categories of every large and mega cap primary listing (2,565 companies) were reviewed against the company's business description. 347 corrections were applied to all listings of each company (same ISIN or name). Examples:
  • Philips and Siemens Healthineers → Medical Devices & Supplies;
  • RELX, Wolters Kluwer → Professional Services;
  • Visa, Mastercard, Adyen, PayPal, Fiserv → Payments & Specialty Finance;
  • Schneider, Legrand, Vestas, Siemens Energy → Electrical Equipment;
  • L'Oréal, Beiersdorf, Estée Lauder → Personal Care;
  • Brookfield Corporation (was Household Products) → Capital Markets;
  • Arista → Networking & Communications Equipment.
• Countries.
  • A country that matched neither the listing's exchange nor its ISIN is corrected where the company's description names the headquarters and a second signal agrees (486 listings, e.g. Malaysian companies marked Japan or Hong Kong). Blank countries were filled the same way (323).
  • Reviewed fixes: Zoetis, Fortis, MercadoLibre, BeOne, Amer Sports and others.
• Ferrovial. Ferrovial SE (registered in the Netherlands since 2023) has one name, country and classification on every line, and its Euronext Amsterdam listing FER.AS (NL0015001FS8) was added.
• Names and summaries. Names that belonged to another company were fixed (Tianqi Lithium, Kingsoft Office, Gazprom Neft, Fox-Wizel). Summaries that describe a different company were blanked.
• Delisted flags.
  • Lines abroad of a share whose home listings are all delisted are delisted too (530 lines, e.g. Electronic Arts and Twitter in Germany).
  • Companies the Nasdaq Trader directory still lists were relisted (Jacobs Solutions, Brink's, Centerspace, several new SPAC units).
  • Two warrants were removed.
• ETFs.
  • A ticker identifies one security on every German exchange, so 4,430 Berlin/Dusseldorf/Munich/Hamburg lines without an ISIN took the ISIN, full name and blank fields of their Xetra or Frankfurt line ("VANG.FTSE A.W. DLA" → "Vanguard FTSE All-World UCITS ETF USD Accumulation").
  • 770 regional lines now carry the categories every other listing of the fund agrees on.
• Cryptocurrencies. Numbered duplicates of a coin (SOL1, DOT1, UNI3, STX1, HNT1) are removed. The weekly update now drops such twins itself.
• Market cap. Siemens Limited (India) is Large Cap, not its parent's Mega Cap.

Sources: the companies' own business descriptions and headquarters in the database, company filings for the reviewed corrections, the Nasdaq Trader symbol directory and Euronext Amsterdam's listing of Ferrovial SE.

Added after review

• Mutual funds removed from the ETF data. Xetra lists the ETFs tradable in Germany, so a Berlin, Dusseldorf, Munich, Hamburg, Hanover, Stuttgart or Frankfurt line without an ISIN whose ticker Xetra doesn't list is a mutual fund (16,786 lines such as "DEKA-BAV FONDS"). Lines whose name says ETF, ETC, ETN or UCITS are kept, except funds of ETFs.
• US delistings, weekly. The US update now compares the NASDAQ, NYSE and NYSE American equities with the Nasdaq Trader symbol directory (scripts/equities/status_model.py):
  • a row the directory no longer lists is delisted;
  • a delisted share it lists again under the same name is relisted;
  • the lines abroad of a share whose home listings are all delisted follow;
  • more than 3% delistings in one run are skipped as a bad download.
    The first run delisted 803 rows (expired SPAC units and warrants, acquired companies, redeemed preferreds), relisted 15 and delisted 77 lines abroad.
• Smaller companies classified. The primary listing of each smaller company without an industry, or whose listings disagree, was classified from its business description. Vague descriptions, shells and diversified holdings stay blank, as do REITs without a stated property type. The result is applied to all of the company's listings: 3,233 newly classified companies and 706 resolved conflicts (6,244 rows), plus 273 blanks filled from the share's other listings.
• More non-shares recognised (check_non_share): redeemable NCDs, rights issues, leveraged products, listed indices, Nordic knock-out and stop-loss certificates, and Swedish subscription warrants (TO3). The 40 such rows are removed.

Still open

• Companies listed on both BSE and NSE (India) get two primary listings, and an ETF can have several (IWDA.AS, IWDA.L, IWDAN.MX).
• Some Malaysian and Japanese summaries describe another company; they were left out of the classification where they disagree with the name.

Checks: full test suite (401 passed), identifier gate, categories check, remove_non_shares.py --dry-run (0 rows), black and ruff.

After merging, CI regenerates compression/; the MCP server needs a redeploy and a cache clear to serve the new data.