Research topic · Evidence and limitations

Reasoning in artificial intelligence: experiments, tests and limits by Elena AILL

Reasoning in AI is not a unique property: it can include inference, causal relationships, composition of multiple facts, planning and verification. This page presents Elena AILL as a documented experimental case, preserving both positive results and failures.

A correct answer, by itself, does not prove that the system followed a general reasoning process. Likewise, a PASS obtained on a limited set does not demonstrate universal ability. For this reason, each measurement reported here remains linked to its phase, its input and the declared gate.

What is reasoning in artificial intelligence

In general terms, reasoning means using information or premises to derive a conclusion while respecting relationships and constraints. The task may require a single step or a sequence: connecting facts, maintaining roles and dependencies, choosing an order of actions, checking whether the outcome satisfies what is required.

However, there is no single test capable of measuring everything. A controlled logic exercise, a causal question, a plan, and a natural language answer elicit different components. The correctness observed must therefore be attributed to the test performed, not automatically extended to every form of AI reasoning.

Inference, causation and planning are not the same thing

Concept	What does he do	How it can be tested	What it doesn't prove
Inference	Derives a conclusion from premises or facts.	Cases with expected outcome and explicit constraints.	General understanding or causality.
Causal reasoning	It deals with cause and effect relationships within the given scope.	Causal questions, checks and targeted comparisons.	Open-domain causality or unproven counterfactuals.
Multi-step / multi-fact	Composes multiple steps or facts while maintaining dependencies and order.	Chains with multiple elements and verifiable answer.	That a simple enumeration constitutes inference.
Planning	Organize facts, propositions, or actions before output.	Structured plans compared to a reference.	Correct execution or verbalization of the plan.
Verification / self-evaluation	Accept, correct, block or refer a candidate.	Gate, negative controls and comparison with the expected.	Guaranteed correctness of each answer.
Text-to-meaning	Converts text into a usable representation.	Round-trip, binding of roles and expected meanings.	The reasoning itself.
Linguistic generation	Transform structures and predicates into a sentence.	Loyalty, order, coverage and absence of inventions.	New knowledge or reasoning.

How Elena represents and verifies a problem

The public phases document different paths, not a single pipeline valid in every situation. In relevant tests the flow can start with linguistic input, move to a representation of meaning, organize facts and relationships, produce a plan, submit it to a verifier, and finally lexicalize the output. Some phases isolate a single trait with meanings oracle or frozen; others try an end-to-end route.

This distinction is causal: if the planner passes the test on a controlled structure but the entire response fails from the free text, the result does not allow us to declare the complete pipeline valid. Furthermore, an experimental candidate is not the same as the promoted runtime. Phase pages specify, when available, which boundary was actually crossed.

Causal, multi-step and multi-fact reasoning

F636 records 17 out of 20 correct cases in its causal set and a positive gate, but the record is “Validated” with no PASS terminal verdict. F637 documents a positive gate for multi-step thought memory while leaving checkpoints unchanged. F879 reports complete measures on direct recall, ownership and anti-forgetting, with paraphrasing above 90%, but remains a limited test.

These experiments show concrete ways of separating relationships, sequences, and composition; they do not demonstrate that the system solves arbitrary causal questions. Also F882, with 0.8 on the practical set and 0.6 on the smoke test, indicates that the result changes with the evaluation perimeter.

Planning and composition of facts

F640 evaluates 92 cases of strategy choice and records an incorrect strategy rate of 0.021739 with a positive gate; the status remains “Validated” without a terminal verdict. F291 documents 8 out of 8 checks for goal reasoning. These measures advocate the use of structures and gates in their context, not master planning.

The role binding, the order of the propositions and the linguistic realization can change the observable outcome while leaving the plan intact. For this reason, planning and sentence composition are evaluated separately.

From F896 failure to F901 repair

This sequence does not say that Elena "has learned to reason". It is a diagnosis followed by corrections and checks of specific components, with different boundaries between reasoning and language.

Phase	Intervention or outcome	Correct border
F896	Structured planner achieves 32/32, but text-to-meaning 3/32 and end-to-end plans/responses 2/28. The reload remains 32/32.	Valid isolated planner; failed natural pipeline.
F897	Fix fact-contract transport: blind multi-fact 20/20, but frozen meaning 14/28 and overall gate FAIL.	Transport and integration, not general reasoning.
F898	Frozen meaning goes from 14/28 to 28/28; blind multi-fact 20/20; topics 28/28; invented predicates 0. The candidate is not promoted.	Mainly lexicalization of predicates.
F899	20/20 speech composition and F897/F898 regressions maintained, without promotion.	Boundary from the plane to the linguistic surface.
F900	Contracts 50/50, verifier and end-to-end 48/50, cold reload passed; the mouth is integrated and promoted.	Language and runtime integration.
F901	Round-trip and cold reload 50/50; passive repair 256/256 versus 102/256 in ablation; full open-domain meaning still 2/100.	Circumscribed text-to-meaning repair.

Phases F897, F899, F900 and F901 are cited by the records verified in the audit but do not have a dedicated public page in the current archive; they are therefore not transformed into non-existent links.

Documented evidence on reasoning

Phase	What head	Outcome	Measure	Limit
F291	Goal reasoning	Source PASS	8/8 checks	Defined set.
F294	Self-evaluation	Source PASS	6/6 checks	It does not guarantee every answer.
F635	Deep reasoning foundation	Validated, without verdict	Documented sets: open, math, commonsense, code, documents, follow-up, and ambiguous	No terminal PASS.
F640	Strategy planner	Validated, without verdict	92 cases; wrong strategy 0.021739	Choice of strategy, not entire pipeline.
F642	Benchmark deep reasoning	Validated, without verdict	60 open and six categories of 40	Candidate not promoted.
F870	Blind testing and reasoning-memory use	Validated, without verdict	Blind passed; usage 0.986	Component use does not equate to general causation.
F871	Candidate versus baseline	PARTIAL	1.0 vs. 0.465; gate yes	The status remains partial.
F884	Reasoning and regression	Validated, without verdict	Overall, reasoning and regression 1.0	Circumscribed set.
F896	Planner and natural pipeline	FAIL	Planner 32/32; end-to-end 2/28	The PASS component does not save the pipeline.

When reasoning fails

Negative results are central because they prevent mistaking a successful component for a complete capability.

F665: total recovery 0.419 and applicable 0.52; the gate does not pass.
F671: direct recall 30/30, but anti-forgetting 3/4; the source verdict is PARTIAL and the overall gate fails.
F689-A: the clean blind reaches 0.8375 and a weakness of the benchmark is recorded, not a general confirmation.
F700: 500 cases and components with score 1 do not change the FAIL terminal status.
F701: stack usage equal to 0; the candidate is not promoted.
F708: variants document shortcuts, ordering issues, and variable binding needs; more attempts are reversed.
F878: reasoning cycle 0.083, evidence support 0, why correct 0 and multi-step 0.2; REVERT_OR_FIX decision.
F880: 12/20, pass rate 0.6, against human reference 20/20.
F896: The isolated planner passes, but the end-to-end pipeline remains 2/28.

These cases block claims such as “solved reasoning,” uniform generalization, or end-to-end reliability. A failure does not prove that every component is useless; However, it demonstrates that the claim must stop at the crossed border.

Reasoning and natural language are not the same thing

A system can represent a relationship correctly and verbalize it poorly; it can also produce a plausible sentence without showing the path that supports it. F896 makes the first case visible: the structured planner passes its set while text-to-meaning and end-to-end output fail. F898-F901 especially improve the linguistic and semantic frontier.

For decoder, lexicalization, text-to-meaning and surface quality, the proprietary reference remains the Linguistic system. This hub uses such results only to explain where the observation of a reasoning path may be blocked or distorted.

How reasoning occurs

A useful test states cases or dataset, expected outcome, metric, and gate. Where available, holdouts or blinds reduce the fit to the known set; negative checks and ablation help understand whether the modified component actually causes the result. A component test answers a different question than an end-to-end test.

F294 exemplifies self-evaluation checks; F874 records 14/14 but does not promote the candidate; F896 explicitly separates planner and pipeline. The Benchmark hub collects methodology and families of tests: a score remains a measurement on the declared perimeter, not a universal test.

Current limitations of the tests

The available evidence depends on the type of input and limited benchmarks. In several places the text-to-meaning representation is a bottleneck; elsewhere a candidate passes a gate without being promoted. “Validated” describes the editorial status of the record and does not automatically equate to PASS. PARTIAL and MIXED are not elevated to terminal success.

End-to-end components and behavior, controlled input and free text, local result and generalization remain distinct. No measure here demonstrates human reasoning, universal understanding, AGI, or consciousness.

The availability of facts over time belongs primarily to the persistent memory; the acquisition and retention of new knowledge belong to continuous learning. They are prerequisites or possible contexts, not synonymous with reasoning.