Ideas Have Genomes: Benchmarking Scientific Lineage Reasoning and Lineage-Grounded Idea Generation

Noosaga Placements

• Artificial Intelligencesubfield95%
  The paper presents a benchmark for evaluating AI systems on scientific lineage reasoning and generation, which falls under general artificial intelligence. The arXiv primary category is cs.AI.
  arXiv:2607.08758v1 [cs.AI]We present IdeaGene-Bench (IG-Bench), a benchmark for scientific lineage reasoning and lineage-grounded idea generation.
• Large Language Modelsframework90%
  The benchmark evaluates LLM-based scientists, and the experiments involve direct LLMs (e.g., GPT-5.5, Claude Opus 4.7) and research agents built on LLMs. The paper explicitly mentions 'LLM-based scientists' and uses LLMs for generation and evaluation.
  Experiments on 14 LLM-based scientists expose a compositional bottleneck.Participants include eight direct LLMs, two GPT-5.5-based research agents, and four CLI harnesses that wrap GPT-5.5 or Claude Opus 4.7
• Evolutionary Computationframework80%
  The paper uses an evolutionary metaphor (inheritance, mutation, selection) and defines six operational evolutionary dynamics (mutation, adaptive radiation, hybridization, speciation, niche competition, isolation) for classifying GenomeDiff patterns. This situates the work within the evolutionary computation framework.
  We define Idea Genome, genome extraction, and GenomeDiff, together with six operational evolutionary dynamics (mutation, adaptive radiation, hybridization, speciation, niche competition, isolation)Evolutionary dynamics classify GenomeDiff patterns into operational categories.
• Natural Language Processingsubfield70%
  The benchmark evaluates LLM-based scientists and involves text-based reasoning and generation from scientific literature, which is closely related to NLP. The paper mentions NLP as one of the 10 scientific domains.
  Experiments on 14 LLM-based scientists expose a compositional bottleneck.10 scientific domains—including NLP [9, 28], computer vision [5, 10, 31], multimodal learning [23, 29], and six additional domains
• Knowledge Representation Reasoningsubfield60%
  The IdeaGene framework involves representing scientific ideas as typed, evidence-grounded objects and reasoning about their inheritance and evolution, which touches on knowledge representation and reasoning.
  IdeaGene framework: each paper or proposal is represented as a set of minimal, typed, evidence-grounded Idea Genome objectsIG-Exam (42 task types, 1,029 instances) tests closed-form lineage reasoning across Idea Genome abstraction, inheritance tracing, evolutionary reasoning, and lineage verification.
• Foundation Modelsframework60%
  The benchmark evaluates LLM-based scientists, which are often considered foundation models. The paper mentions 'LLM-based auto-research systems' and evaluates models like GPT-5.5 and Claude Opus 4.7, which are large-scale foundation models.
  LLM-based auto-research systems now search literature, synthesize hypotheses, run experiments, and write paper-like reportsParticipants include eight direct LLMs, two GPT-5.5-based research agents

Abstract

Scientific ideas rarely start from a blank page. They inherit mechanisms, repair known limitations, and recombine pieces of earlier work, much like biological genomes. Current benchmarks still say little about whether AI systems can follow this inheritance structure. We present IdeaGene-Bench (IG-Bench), a benchmark for scientific lineage reasoning and lineage-grounded idea generation. IG-Bench is organized around the IdeaGene framework: each paper or proposal is represented as a set of minimal, typed, evidence-grounded Idea Genome objects, and a GenomeDiff aligns these objects to record inheritance, mutation, loss, external import, and novel insertion under six operational evolutionary dynamics. The benchmark contains 1,961 golden lineage traces, 1,085 curated Idea Genome objects, and 920 pairwise GenomeDiff records across 10 scientific domains. It supports two evaluations. IG-Exam (42 task types, 1,029 instances) tests closed-form lineage reasoning across Idea Genome abstraction, inheritance tracing, evolutionary reasoning, and lineage verification. IG-Arena evaluates generation with a lineage-conditioned Population-Evolution Score(PES), asking whether a proposal can be inserted as a coherent descendant of a given lineage population: it should inherit the right Idea Genome objects, vary meaningfully from nearby work, and offer selection value for future research. Experiments on 14 LLM-based scientists expose a compositional bottleneck. The strongest system reaches only 27.3% exact accuracy on lineage reasoning, and structured lineage context reshuffles system rankings rather than helping every participant uniformly.

Paper Context