Panel-design run fails after CZI step: irrelevant CZI matches followed by `parsing error...`

[arashabadi]

Thank you again for sharing SpatialAgent.

I’m trying to design a mouse lung influenza panel, but I have not yet had a successful end-to-end run.
The run reaches the CZI step, returns mostly irrelevant datasets, and then terminates with parsing error....

• Task: 100-gene panel for C57BL/6J mouse lung.
• CZI metadata is available and data_path is correctly set. (Improve notebook UX outside repo root: default data_path causes false CZI metadata missing errors #3 resolved) ... So this is not the previous missing-file issue.

Observed behavior

CZI search executes:

search_czi_datasets({"query": "lung, Mus musculus", "n_datasets": 5})

but top matches can be non-lung tissues (e.g., embryo / skin). Shortly after that step, the agent prints parsing error....

Run stops early; no reliable final panel is produced. but For mouse + lung intent, top CZI results should be lung-relevant mouse datasets and the run should continue normally after CZI retrieval (no parser termination).

Suspected causes

• search_czi_datasets appears to rank embeddings over broad metadata without strict pre-filtering by organism/tissue.
• CZI tool output may be oversized/noisy (print + return duplication), which can increase parser fragility.

Must Fix (I think)

1. Add structured filters in search_czi_datasets (before ranking):
   • organism, tissue, optional disease, strict=True.
   • Filter dataframe first, then rank within filtered rows.
2. Add deterministic fallback when strict filters return no rows:
   • Controlled relaxation + clear warning.
3. Remove duplicate large output payload:
   • Avoid both print(output) and return output for same content.
4. Keep CZI output compact to reduce parser risk.

Should Fix

5. Add ranking guardrails:
   • Strong penalty for organism mismatch.
   • Boost exact/partial tissue match (lung, respiratory).
6. Update panel-design skill to prefer structured CZI constraints over free-text-only queries.

[hansen7]

thx, could you share the agent's trace so I can reproduce the issue?

Also, which embedding model are you using? After testing, we recommend either OpenAI's text-embedding-3-small or locally served Qwen/Qwen3-Embedding-0.6B — other local models (e.g., PubMedBERT, BGE-large) had issues matching cell types like T cells and B cells correctly.

[arashabadi]

Here is the trace from my failing run:

<user query>
Design a 100-gene panel for C57BL/6J mouse lung tissue that captures all lung cell types, from stromal to immune. The study is focused on influenza infection (both primary infection and rechallenge), and the goal is to detect all cells present on the Xenium slide.
</user query>

<skill> retrieved panel_design </skill>

<skill-tools> query_celltype_genesets; search_czi_datasets; extract_czi_markers; search_panglao; validate_genes_expression; query_pubmed; query_tissue_expression </skill-tools>

<tool> selected execute_python; query_celltype_genesets; execute_bash; inspect_tool_code; search_czi_datasets; web_search; extract_czi_markers; search_panglao; validate_genes_expression; query_pubmed; query_tissue_expression; report_subagent; fetch_supplementary_from_doi; scanpy_score_genes; extract_url_content; search_semantic_scholar; aggregate_gene_voting; extract_pdf_content; download_czi_reference; summarize_celltypes; query_disease_genes; search_cellmarker2 </tool>

I will design a 100-gene panel for C57BL/6J mouse lung, covering epithelial, stromal, endothelial, and immune compartments, with attention to influenza infection contexts.

First, I will search for a suitable mouse lung reference dataset in the CZI CELLxGENE Census to identify marker genes.

<act>
search_czi_datasets({"query": "lung, Mus musculus", "n_datasets": 5})
</act>

<observation>
Output:
CZI CELLxGENE Search Results for 'lung, Mus musculus':

Dataset 1:
  dataset_id: b47eaa46-508d-4817-8a75-5bece9ea30f9
  dataset_title: Whole dataset: Normalized subset 4
  collection_name: A single-cell transcriptional timelapse of mouse embryonic development, from gastrula to pup
  organism: Mus musculus
  tissue: embryo
  disease: normal
  similarity_score: 0.772

Dataset 2:
  dataset_id: e6a47f6f-ae9a-41ff-a6f7-5f9b9dbd3e7f
  dataset_title: Whole dataset: Normalized subset 3
  collection_name: A single-cell transcriptional timelapse of mouse embryonic development, from gastrula to pup
  organism: Mus musculus
  tissue: embryo
  disease: normal
  similarity_score: 0.772

Dataset 3:
  dataset_id: a91d5064-aba3-4688-9fcd-6187b2f435a4
  dataset_title: Whole dataset: Normalized subset 1
  collection_name: A single-cell transcriptional timelapse of mouse embryonic development, from gastrula to pup
  organism: Mus musculus
  tissue: embryo
  disease: normal
  similarity_score: 0.771

Dataset 4:
  dataset_id: 7be2824e-d62a-42ef-9169-084555c6df87
  dataset_title: all
  collection_name: Single cell RNA sequencing of paw skin from healthy and Col7a1 knockout (RDEB) mice
  organism: Mus musculus
  tissue: autopod skin
  disease: epidermolysis bullosa;normal
  similarity_score: 0.770

Dataset 5:
  dataset_id: 0bce33ed-455c-4e12-93f8-b7b04a2de4a1
  dataset_title: Whole dataset: Normalized subset 2
  collection_name: A single-cell transcriptional timelapse of mouse embryonic development, from gastrula to pup
  organism: Mus musculus
  tissue: embryo
  disease: normal
  similarity_score: 0.769


Result: "CZI CELLxGENE Search Results for 'lung, Mus musculus':\n\nDataset 1:\n  dataset_id: b47eaa46-508d-4817-8a75-5bece9ea30f9\n  dataset_title: Whole dataset: Normalized subset 4\n  collection_name: A single-cell transcriptional timelapse of mouse embryonic development, from gastrula to pup\n  organism: Mus musculus\n  tissue: embryo\n  disease: normal\n  similarity_score: 0.772\n\nDataset 2:\n  dataset_id: e6a47f6f-ae9a-41ff-a6f7-5f9b9dbd3e7f\n  dataset_title: Whole dataset: Normalized subset 3\n  collection_name: A single-cell transcriptional timelapse of mouse embryonic development, from gastrula to pup\n  organism: Mus musculus\n  tissue: embryo\n  disease: normal\n  similarity_score: 0.772\n\nDataset 3:\n  dataset_id: a91d5064-aba3-4688-9fcd-6187b2f435a4\n  dataset_title: Whole dataset: Normalized subset 1\n  collection_name: A single-cell transcriptional timelapse of mouse embryonic development, from gastrula to pup\n  organism: Mus musculus\n  tissue: embryo\n  disease: normal\n  similarity_score: 0.771\n\nDataset 4:\n  dataset_id: 7be2824e-d62a-42ef-9169-084555c6df87\n  dataset_title: all\n  collection_name: Single cell RNA sequencing of paw skin from healthy and Col7a1 knockout (RDEB) mice\n  organism: Mus musculus\n  tissue: autopod skin\n  disease: epidermolysis bullosa;normal\n  similarity_score: 0.770\n\nDataset 5:\n  dataset_id: 0bce33ed-455c-4e12-93f8-b7b04a2de4a1\n  dataset_title: Whole dataset: Normalized subset 2\n  collection_name: A single-cell transcriptional timelapse of mouse embryonic development, from gastrula to pup\n  organism: Mus musculus\n  tissue: embryo\n  disease: normal\n  similarity_score: 0.769"
</observation>

parsing error...

Cost Summary (gemini-3-pro-preview)
Total calls:    4
Input tokens:   24,891
Output tokens:  211
Total tokens:   25,102
Total cost:     $0.0523

Also, the embedding model used in this run was: local qwen3-0.6b (Qwen/Qwen3-Embedding-0.6B)

I did not configure embedding-specific environment variables in the notebook session (only GEMINI_API_KEY was set), so SpatialAgent used its default embedding path. In this codebase, if USE_LOCAL_EMBEDDINGS is not explicitly set to "false", embeddings default to local mode, and the default local embedding model is qwen3-0.6b.

Based on the trace, there are two likely contributors (suggested by GPT-5.3-Codex):

1. Retrieval relevance issue (upstream quality problem):
   The CZI search appears to rank over broad metadata embeddings without strict metadata pre-filtering (organism/tissue constraints applied before ranking). For a query like "lung, Mus musculus", this can still rank irrelevant mouse datasets (e.g., embryo/skin) above lung datasets. This explains why the top matches were biologically off-target.

2. Parser failure issue (immediate termination trigger): [Most probable cause]
   The same CZI content appears twice in the observation payload ("Output" plus a duplicated long "Result" string). That duplication significantly increases observation size/noise. Immediately after this, the model produced a malformed/invalid next turn (missing required act/conclude tags), which triggered the parser recovery path and then termination.