WISE- Wired Intelligence for Semantic Encoding

An Adaptive Learning Model architecture that outperforms large language models !

In a world dominated by Large Language Models (LLMs) that heavily rely on extensive training to discern patterns, we proudly introduce a revolutionary architecture:

Wired Intelligence for Semantic Encoding (WISE).

WISE distinguishes itself by employing a series of specialized models to process sequences of tokens, which are segmented to enhance semantic coherence. Unlike conventional models, WISE architecture adeptly maintains contextual awareness without relying on the attention mechanism, offering a nuanced understanding of language.

We believe that certain challenges linked to transformer-based language models—specifically, hallucinations, inconsistencies and reliability issues, and difficulties with solving math problems—are fundamental characteristics of the attention mechanism. LLMs have achieved remarkable feats, but at their core, they remain pattern followers. They generate outputs based on vast amounts of pre-existing data they've been trained on, often replicating patterns without true comprehension. These models, while advanced, still aren't "thinking" entities. They provide answers based on patterns they've seen, rather than genuine understanding. This limits language models to excel at reasoning. We believe that the key to achieving reasoning capabilities lies in enabling models to engage in a "thought loop" as they process information. This approach allows models to reflect and build upon their initial interpretations, enhancing their ability to reason and understand complex concepts. Creating a thought loop involves teaching models to identify moments to pause, reflect, verify, and validate their understanding. By breaking down a problem into smaller segments and associating a cycle of reflection with each segment, we can enable models to enhance their reasoning capabilities. This method fosters a deeper and more nuanced understanding, allowing models to approach complex problems with greater precision and insight. WISE utilizes innovative methods to segment problems and generate thought loops, which makes it very good at reasoning and math.

How do we segment a sequence of tokens?

When a sequence of tokens is passed into our architecture, it is first processed by a pre-trained classifier, which segments the sequence by semantic coherence.

 Then, these segments are passed into another pre-trained sequence-to-sequence model, which transforms them into information pairs. Each segment is fed back to create different pairs. 

Then, these pairs are fed into a feedforward network, which adjusts its weights based on the data and creates meaningful, classifiable constructs. The constructs are saved in parametric memory, while the information pairs are stored in non-parametric memory.

Inference 

During inference, the classified construct is passed into a sequence-to-sequence model, which manipulates the tokens to generate responses.

Examples : 

Following are the advantages of this approach

1- Improved factual accuracy: 

Each segment of the response created during inference initiates another loop of thoughts, ensuring accuracy.

2- Better at math and reasoning: 

Each segment of the response created during inference generates another loop of thoughts. This allows it to create solutions not just from the examples it was trained on but also by combining multiple examples. 

3- No context window:

 Even though the ALM processes each segment individually, since each segment creates a meaningful chain of thoughts, it allows for the retention of necessary information from previous segments.

Car has four wheels. Bike has two wheels . Trike has three wheels. Bus has four wheels . Group the vehicles with four wheels .

4- Better at instruction following: 

Before giving access to gardener , verify by asking for the secret code (1234) 

5- Better at asking questions: 

Each loop of thoughts generated by each segment should conclude without leaving any questions unanswered. Therefore, if the architecture encounters any unfamiliar thoughts, the WISE will pose a question to  us. This enables the WISE to function as a general brain in automating business SOPs.

Before giving access to the gardener , verify that it's the gardener. 

6- Generative capabilities: 

Although the ALM cannot create novel or non-obvious stories and other creative works, it excels at combining patterns from several examples it has encountered, without compromising factual accuracy. This capability will enable the ALM to become a valuable tool for creating business documents.