Large language models (LLMs) built using transformer architectures heavily depend on pre-training with large-scale data to predict sequential tokens. This complex and resource-intensive process requires enormous computational infrastructure and well-constructed data pipelines. The growing demand for efficient and accessible LLMs has led researchers to explore techniques that balance resource use and performance, emphasizing achieving competitive results without relying on industry-scale resources.

Developing LLMs is filled with challenges, especially regarding computation and data efficiency. Pre-training models with billions of parameters demand advanced techniques and substantial infrastructure. High-quality data and robust training methods are crucial, as models face gradient instability and performance degradation during training. Open-source LLMs often struggle to match proprietary counterparts because of limited access to computational power and high-caliber datasets. Therefore, the challenge lies in creating efficient and high-performing models, enabling smaller research groups to participate actively in advancing AI technology. Solving this problem necessitates innovation in data handling, training stabilization, and architectural design.

Existing research in LLM training emphasizes structured data pipelines, using techniques like data cleaning, dynamic scheduling, and curriculum learning to improve learning outcomes. However, stability remains a persistent issue. Large-scale training is susceptible to gradient explosions, loss spikes, and other technical difficulties, requiring careful optimization. Training long-context models introduce additional complexity as attention mechanisms’ computational demands grow quadratically with sequence length. Existing approaches like advanced optimizers, initialization strategies, and synthetic data generation help alleviate these issues but often fall short when scaled to full-sized models. The need for scalable, stable, and efficient methods in LLM training is more urgent than ever.

Researchers at the Gaoling School of Artificial Intelligence, Renmin University of China, developed YuLan-Mini. With 2.42 billion parameters, this language model improves computational efficiency and performance with data-efficient methods. By leveraging publicly available data and focusing on data-efficient training techniques, YuLan-Mini achieves remarkable performance comparable to larger industry models.

YuLan-Mini’s architecture incorporates several innovative elements to enhance training efficiency. Its decoder-only transformer design employs embedding tying to reduce parameter size and improve training stability. The model uses Rotary Positional Embedding (ROPE) to handle long contexts effectively, extending its context length to 28,672 tokens, an advancement over typical models. Other key features include SwiGLU activation functions for better data representation and a carefully designed annealing strategy that stabilizes training while maximizing learning efficiency. Synthetic data was critical, supplementing the 1.08 trillion tokens of training data sourced from open web pages, code repositories, and mathematical datasets. These features enable YuLan-Mini to deliver robust performance with a limited computing budget.

YuLan-Mini’s performance achieved scores of 64.00 on HumanEval in zero-shot scenarios, 37.80 on MATH-500 in four-shot settings, and 49.10 on MMLU in five-shot tasks. These results underscore its competitive edge, as the model’s performance is comparable to much larger and resource-intensive counterparts. The innovative context length extension to 28K tokens allowed YuLan-Mini to excel in long-text scenarios while still maintaining high accuracy in short-text tasks. This dual capability sets it apart from many existing models, which often sacrifice one for the other.

Key takeaways from the research include:

• Using a meticulously designed data pipeline, YuLan-Mini reduces reliance on massive datasets while ensuring high-quality learning.
• Techniques like systematic optimization and annealing prevent common issues like loss spikes and gradient explosions.
• Extending the context length to 28,672 tokens enhances the model’s applicability to complex, long-text tasks.
• Despite its modest computational requirements, YuLan-Mini achieves results comparable to those of much larger models, demonstrating the effectiveness of its design.
• The integration of synthetic data improves training outcomes and reduces the need for proprietary datasets.

In conclusion, YuLan-Mini is a great new addition to evolving efficient LLMs. Its ability to deliver high performance with limited resources addresses critical barriers to AI accessibility. The research team’s focus on innovative techniques, from data efficiency to training stability, highlights the potential for smaller-scale research to contribute to the field significantly. With just 1.08T tokens, YuLan-Mini sets a benchmark for resource-efficient LLMs.

Check out the Paper and GitHub Page. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. Don’t Forget to join our 60k+ ML SubReddit.

Trending: LG AI Research Releases EXAONE 3.5: Three Open-Source Bilingual Frontier AI-level Models Delivering Unmatched Instruction Following and Long Context Understanding for Global Leadership in Generative AI Excellence….

The post YuLan-Mini: A 2.42B Parameter Open Data-efficient Language Model with Long-Context Capabilities and Advanced Training Techniques appeared first on MarkTechPost.

The semiconductor industry enables advancements in consumer electronics, automotive systems, and cutting-edge computing technologies. The production of semiconductors involves sophisticated processes that demand unparalleled precision and expertise. These processes include chip design, manufacturing, testing, and optimization, each stage requiring deep domain knowledge. The field has traditionally depended on seasoned engineers whose experience has been built over decades. However, the industry faces a significant challenge: the rapid retirement of veteran experts, creating a knowledge gap that threatens innovation and efficiency. This growing concern has prompted companies to explore AI as a viable solution for capturing, scaling, and leveraging expert knowledge. Also, the cost and time associated with chip design and manufacturing must be minimized to meet market demands. These challenges highlight the limitations of traditional methods and emphasize the necessity of tailored AI solutions.

Existing approaches to these challenges include generalized AI models and basic automation tools. While these methods have been beneficial in analyzing data and improving decision-making, they often fall short in addressing the unique complexities of the semiconductor industry. General-purpose AI tools, for instance, lack the domain-specific understanding required to analyze intricate manufacturing processes effectively. As a result, companies cannot fully bridge the gap between theoretical AI capabilities and practical industry needs, leaving room for specialized solutions to transform the field.

Researchers from Meta, AITOMATIC, and other collaborators under the Foundation Models workgroup of the AI Alliance have introduced SemiKong. SemiKong represents the world’s first semiconductor-focused large language model (LLM), designed using the Llama 3.1 platform. This model was fine-tuned with extensive semiconductor-specific datasets, including industry documents, research papers, and anonymized operational data. Unlike generic AI systems, SemiKong is tailored to understand semiconductor processes’ unique terminology and requirements. By integrating this model with the AITOMATIC Domain-Expert Agents (DXAs), companies can effectively leverage AI tools to address specific industry challenges. These innovations aim to reduce costs, accelerate development timelines, and promote collaboration across the semiconductor sector.

The technology behind SemiKong is built on advanced AI and neurosymbolic architectures. AITOMATIC’s DXAs operate through a structured three-phase lifecycle: 

1. Capturing domain expertise
2. Training the model with synthetic and structured data
3. Applying the resulting system in real-world scenarios 

SemiKong plays a central role in this ecosystem, acting as the “brain” for complex reasoning and decision-making tasks. Lightweight model versions, such as Llama 3.2, complement the main system by enabling faster data access and analysis in resource-constrained environments. These models integrate seamlessly with manufacturing systems and IoT platforms, allowing companies to optimize workflows, predict maintenance needs, and improve decision-making.

SemiKong has outperformed several closed-source language models in generating semiconductor-specific content and understanding complex processes. This has led to tangible benefits, including a 20-30% reduction in time to market for new chip designs and a 15-25% improvement in first-time-right manufacturing outcomes. These tools have also improved the onboarding process for new engineers, accelerating their learning curve by 40-50%. In one example, SemiKong-enabled DXAs reduced the time required for etching recipe formulation, which typically takes hours to minutes.

The key takeaways from the research underscore the significance of SemiKong and DXAs in the semiconductor field:

1. DXAs effectively capture and structure the knowledge of veteran engineers, ensuring that critical expertise is preserved and scaled for future use.  
2. SemiKong reduces chip design time-to-market by up to 30%, significantly cutting costs and improving operational efficiency.  
3. By simplifying and expediting the onboarding process, DXAs help new engineers become productive faster, reducing the industry’s reliance on seasoned experts.  
4. Integrating IoT platforms enables real-time parameter calibration and predictive maintenance, enhancing equipment performance and reliability.

In conclusion, the research highlights a pioneering solution to one of the semiconductor industry’s most pressing challenges: the loss of critical domain expertise. By introducing SemiKong and DXAs, the researchers have provided a comprehensive framework that preserves knowledge and enhances productivity and innovation. These advancements can potentially reshape semiconductor manufacturing, offering scalable, cost-effective solutions to address the field’s complexities. Integrating AI tools like SemiKong is crucial for a more efficient and resilient semiconductor industry.

Check out the Details and GitHub Page. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. Don’t Forget to join our 60k+ ML SubReddit.

Trending: LG AI Research Releases EXAONE 3.5: Three Open-Source Bilingual Frontier AI-level Models Delivering Unmatched Instruction Following and Long Context Understanding for Global Leadership in Generative AI Excellence….

The post Meet SemiKong: The World’s First Open-Source Semiconductor-Focused LLM appeared first on MarkTechPost.

The field of Natural Language Processing (NLP) has made significant strides with the development of large-scale language models (LLMs). However, this progress has brought its own set of challenges. Training and inference require substantial computational resources, the availability of diverse, high-quality datasets is critical, and achieving balanced utilization in Mixture-of-Experts (MoE) architectures remains complex. These factors contribute to inefficiencies and increased costs, posing obstacles to scaling open-source models to match proprietary counterparts. Moreover, ensuring robustness and stability during training is an ongoing issue, as even minor instabilities can disrupt performance and necessitate costly interventions.

DeepSeek-AI just gave a Christmas present to the AI world by releasing DeepSeek-V3, a Mixture-of-Experts (MoE) language model featuring 671 billion parameters, with 37 billion activated per token. The model builds on proven architectures such as Multi-Head Latent Attention (MLA) and DeepSeekMoE, which were refined in earlier versions. DeepSeek-V3 has been trained on an extensive dataset of 14.8 trillion high-quality tokens, ensuring a broad and diverse knowledge base. Importantly, the model is fully open-source, with accessible models, papers, and training frameworks for the research community to explore.

Technical Details and Benefits

DeepSeek-V3 incorporates several innovations aimed at addressing long-standing challenges in the field. Its auxiliary-loss-free load balancing strategy ensures efficient distribution of computational loads across experts while maintaining model performance. The adoption of a multi-token prediction training objective enhances data efficiency and facilitates faster inference through speculative decoding. Additionally, FP8 mixed precision training improves computational efficiency by reducing GPU memory usage without sacrificing accuracy. The DualPipe algorithm further minimizes pipeline bubbles by overlapping computation and communication phases, reducing all-to-all communication overhead. These advancements enable DeepSeek-V3 to process 60 tokens per second during inference—a significant improvement over its predecessor.

Performance Insights and Results

DeepSeek-V3 has been rigorously evaluated across multiple benchmarks, demonstrating strong performance. On educational datasets like MMLU and MMLU-Pro, it achieved scores of 88.5 and 75.9, respectively, outperforming other open-source models. In mathematical reasoning tasks, it set new standards with a score of 90.2 on MATH-500. The model also performed exceptionally in coding benchmarks such as LiveCodeBench. Despite these achievements, the training cost was kept relatively low at $5.576 million, requiring only 2.788 million H800 GPU hours. These results highlight DeepSeek-V3’s efficiency and its potential to make high-performance LLMs more accessible.

Conclusion

DeepSeek-V3 represents a meaningful advancement in open-source NLP research. By tackling the computational and architectural challenges associated with large-scale language models, it establishes a new benchmark for efficiency and performance. Its innovative training methods, scalable architecture, and strong evaluation results make it a competitive alternative to proprietary models. DeepSeek-AI’s commitment to open-source development ensures that the broader research community can benefit from its advancements.

Check out the Paper, GitHub Page, and Model on Hugging Face. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. Don’t Forget to join our 60k+ ML SubReddit.

Trending: LG AI Research Releases EXAONE 3.5: Three Open-Source Bilingual Frontier AI-level Models Delivering Unmatched Instruction Following and Long Context Understanding for Global Leadership in Generative AI Excellence….

The post DeepSeek-AI Just Released DeepSeek-V3: A Strong Mixture-of-Experts (MoE) Language Model with 671B Total Parameters with 37B Activated for Each Token appeared first on MarkTechPost.

Autoregressive (AR) models have changed the field of image generation, setting new benchmarks in producing high-quality visuals. These models break down the image creation process into sequential steps, each token generated based on prior tokens, creating outputs with exceptional realism and coherence. Researchers have widely adopted AR techniques for computer vision, gaming, and digital content creation applications. However, the potential of AR models is often constrained by their inherent inefficiencies, particularly their slow generation process, which remains a significant hurdle in real-time applications.

Among many concerns, a critical one that AR models face is their speed. The token-by-token generation process is inherently sequential, meaning each new token must wait for its predecessor to complete. This approach limits scalability and results in high latency during image generation tasks. For instance, generating a 256×256 image using traditional AR models like LlamaGen requires 256 steps, translating to approximately five seconds on modern GPUs. Such delays hinder their deployment in applications that demand instantaneous results. Also, while AR models excel in maintaining the fidelity of their outputs, they struggle to meet the growing demand for both speed and quality in large-scale implementations.

Efforts to accelerate AR models have yielded various methods, such as predicting multiple tokens simultaneously or adopting masking strategies during generation. These approaches aim to reduce the required steps but often compromise the quality of the generated images. For example, in multi-token generation techniques, the assumption of conditional independence among tokens introduces artifacts, undermining the cohesiveness of the output. Similarly, masking-based methods allow for faster generation by training models to predict specific tokens based on others, but their effectiveness diminishes when generation steps are drastically reduced. These limitations highlight the need for a new approach to enhance AR model efficiency.

Tsinghua University and Microsoft Research researchers have introduced a solution to these challenges: Distilled Decoding (DD). This method builds on flow matching, a deterministic mapping that connects Gaussian noise to the output distribution of pre-trained AR models. Unlike conventional methods, DD does not require access to the original training data of the AR models, making it more practical for deployment. The research demonstrated that DD can transform the generation process from hundreds of steps to as few as one or two while preserving the quality of the output. For example, on ImageNet-256, DD achieved a speed-up of 6.3x for VAR models and an impressive 217.8x for LlamaGen, reducing generation steps from 256 to just one.

The technical foundation of DD is based on its ability to create a deterministic trajectory for token generation. Using flow matching, DD maps noisy inputs to tokens to align their distribution with the pre-trained AR model. During training, the mapping is distilled into a lightweight network that can directly predict the final data sequence from a noise input. This process ensures faster generation and provides flexibility in balancing speed and quality by allowing intermediate steps when needed. Unlike existing methods, DD eliminates the trade-off between speed and fidelity, enabling scalable implementations across diverse tasks.

In experiments, DD highlights its superiority over traditional methods. For instance, using VAR-d16 models, DD achieved one-step generation with an FID score increase from 4.19 to 9.96, showcasing minimal quality degradation despite a 6.3x speed-up. For LlamaGen models, the reduction in steps from 256 to one resulted in an FID score of 11.35, compared to 4.11 in the original model, with a remarkable 217.8x speed improvement. DD demonstrated similar efficiency in text-to-image tasks, reducing generation steps from 256 to two while maintaining a comparable FID score of 28.95 against 25.70. The results underline DD’s ability to drastically enhance speed without significant loss in image quality, a feat unmatched by baseline methods.

Several key takeaways from the research on DD include:

1. DD reduces generation steps by orders of magnitude, achieving up to 217.8x faster generation than traditional AR models.
2. Despite the accelerated process, DD maintains acceptable quality levels, with FID score increases remaining within manageable ranges.
3. DD demonstrated consistent performance across different AR models, including VAR and LlamaGen, regardless of their token sequence definitions or model sizes.
4. The approach allows users to balance quality and speed by choosing one-step, two-step, or multi-step generation paths based on their requirements.
5. The method eliminates the need for the original AR model training data, making it feasible for practical applications in scenarios where such data is unavailable.
6. Due to its efficient distillation approach, DD can potentially impact other domains, such as text-to-image synthesis, language modeling, and image generation.

In conclusion, with the introduction of Distilled Decoding, researchers have successfully addressed the longstanding speed-quality trade-off that has plagued AR generation processes by leveraging flow matching and deterministic mappings. The method accelerates image synthesis by reducing steps drastically and preserves the outputs’ fidelity and scalability. With its robust performance, adaptability, and practical deployment advantages, Distilled Decoding opens new frontiers in real-time applications of AR models. It sets the stage for further innovation in generative modeling.

Check out the Paper and GitHub Page. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. Don’t Forget to join our 60k+ ML SubReddit.

Trending: LG AI Research Releases EXAONE 3.5: Three Open-Source Bilingual Frontier AI-level Models Delivering Unmatched Instruction Following and Long Context Understanding for Global Leadership in Generative AI Excellence….

The post Microsoft and Tsinghua University Researchers Introduce Distilled Decoding: A New Method for Accelerating Image Generation in Autoregressive Models without Quality Loss appeared first on MarkTechPost.

Graphical User Interfaces (GUIs) are central to how users engage with software. However, building intelligent agents capable of effectively navigating GUIs has been a persistent challenge. The difficulties arise from the need to understand visual context, accommodate dynamic and varied GUI designs, and integrate these systems with language models for intuitive operation. Traditional methods often struggle with adaptability, especially in handling complex layouts or frequent changes in GUIs. These limitations have slowed progress in automating GUI-related tasks, such as software testing, accessibility enhancements, and routine task automation.

Researchers from Tsinghua University have just open-sourced and introduced CogAgent-9B-20241220, the latest version of CogAgent. CogAgent is an open-source GUI agent model powered by Visual Language Models (VLMs). This tool addresses the shortcomings of conventional approaches by combining visual and linguistic capabilities, enabling it to navigate and interact with GUIs effectively. CogAgent features a modular and extensible design, making it a valuable resource for both developers and researchers. Hosted on GitHub, the project promotes accessibility and collaboration within the community.

At its core, CogAgent interprets GUI components and their functionalities by leveraging VLMs. By processing both visual layouts and semantic information, it can execute tasks like clicking buttons, entering text, and navigating menus with precision and reliability.

Technical Details and Benefits

CogAgent’s architecture is built on advanced VLMs, optimized to handle both visual data, such as screenshots, and textual information simultaneously. It incorporates a dual-stream attention mechanism that maps visual elements (e.g., buttons and icons) to their textual labels or descriptions, enhancing its ability to predict user intent and execute relevant actions.

One of the standout features of CogAgent is its capacity to generalize across a wide variety of GUIs without requiring extensive retraining. Transfer learning techniques enable the model to adapt quickly to new layouts and interaction patterns. Additionally, it integrates reinforcement learning, allowing it to refine its performance through feedback. Its modular design supports seamless integration with third-party tools and datasets, making it versatile for different applications.

The benefits of CogAgent include:

• Improved Accuracy: By integrating visual and linguistic cues, the model achieves higher precision compared to traditional GUI automation solutions.
• Flexibility and Scalability: Its design allows it to work across diverse industries and platforms with minimal adjustments.
• Community-Driven Development: As an open-source project, CogAgent fosters collaboration and innovation, encouraging a broader range of applications and improvements.

Results and Insights

Evaluations of CogAgent highlight its effectiveness. According to its technical report, the model achieved leading performance in benchmarks for GUI interaction. For example, it excelled in automating software navigation tasks, surpassing existing methods in both accuracy and speed. Testers noted its ability to manage complex layouts and challenging scenarios with remarkable competence.

Additionally, CogAgent demonstrated significant efficiency in data usage. Experiments revealed that it required up to 50% fewer labeled examples compared to traditional models, making it cost-effective and practical for real-world deployment. It further enhanced its adaptability and performance over time, as the model learned from user interactions and specific application contexts.

Conclusion

CogAgent offers a thoughtful and practical solution to longstanding challenges in GUI interaction. By combining the strengths of Visual Language Models with a user-focused design, researchers at Tsinghua University have created a tool that is both effective and accessible. Its open-source nature ensures that the broader community can contribute to its growth, unlocking new possibilities for software automation and accessibility. As an innovation in GUI interaction, CogAgent marks a step forward in creating intelligent, adaptable agents that can meet diverse user needs.

Check out the Technical Report and GitHub Page. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. Don’t Forget to join our 60k+ ML SubReddit.

Trending: LG AI Research Releases EXAONE 3.5: Three Open-Source Bilingual Frontier AI-level Models Delivering Unmatched Instruction Following and Long Context Understanding for Global Leadership in Generative AI Excellence….

The post Tsinghua University Researchers Just Open-Sourced CogAgent-9B-20241220: The Latest Version of CogAgent appeared first on MarkTechPost.

Training large-scale AI models such as transformers and language models have become an indispensable yet highly demanding process in AI. With billions of parameters, these models offer groundbreaking capabilities but come at a steep cost in terms of computational power, memory, and energy consumption. For example, OpenAI’s GPT-3 comprises 175 billion parameters and requires weeks of GPU training. Such massive requirements limit these technologies to organizations with substantial computational resources, exacerbating concerns over energy efficiency and environmental impact. Addressing these challenges has become critical to ensuring the broader accessibility and sustainability of AI advancements.

The inefficiencies in training large models stem primarily from their reliance on dense matrices, which demand significant memory and computing power. The limited support for optimized low-precision or low-rank operations in modern GPUs further compounds these requirements. While some methods, such as matrix factorization and heuristic rank reduction, have been proposed to alleviate these issues, their real-world applicability is constrained. For instance, GaLore enables training on single-batch settings but suffers from impractical runtime overhead. Similarly, LTE, which adopts low-rank adapters, struggles with convergence on large-scale tasks. The lack of a method that simultaneously reduces memory usage, computational cost, and training time without compromising performance has created an urgent need for innovative solutions.

Researchers from the University at Albany SUNY, the University of California at Santa Barbara, Amazon Alexa AI, and Meta introduced Computing-and Memory-Efficient training method via Rank-Adaptive tensor optimization (CoMERA), a novel framework that combines memory efficiency with computational speed through rank-adaptive tensor compression. Unlike traditional methods focusing solely on compression, CoMERA adopts a multi-objective optimization approach to balance compression ratio and model accuracy. It utilizes tensorized embeddings and advanced tensor-network contractions to optimize GPU utilization, reducing runtime overhead while maintaining robust performance. The framework also introduces CUDA Graph to minimize kernel-launching delays during GPU operations, a significant bottleneck in traditional tensor compression approaches.

CoMERA’s foundation is based on adaptive tensor representations, which allow model layers to adjust their ranks dynamically based on resource constraints. By modifying tensor ranks, the framework achieves compression without compromising the integrity of neural network operations. This dynamic optimization is achieved through a two-stage training process: 

1. An early stage focused on stable convergence 
2. A late stage that fine-tunes ranks to meet specific compression targets

In a six-encoder transformer model, CoMERA achieved compression ratios ranging from 43x in its early stage to an impressive 361x in its late-stage optimizations. Also, it reduced memory consumption by 9x compared to GaLore, with 2-3x faster training per epoch.

When applied to transformer models trained on the MNLI dataset, CoMERA reduced model sizes from 256 MB to as little as 3.2 MB while preserving accuracy. In large-scale recommendation systems like DLRM, CoMERA compressed models by 99x and achieved a 7x reduction in peak memory usage. The framework also excelled in pre-training CodeBERT, a domain-specific large language model, where it gained a 4.23x overall compression ratio and demonstrated a 2x speedup during certain training phases. These results underscore its ability to handle diverse tasks and architectures, extending its applicability across domains.

The key takeaways from this research are as follows:

• CoMERA achieved compression ratios of up to 361x for specific layers and 99x for full models, drastically reducing storage and memory requirements.
• The framework delivered 2-3x faster training times per epoch for transformers and recommendation systems, saving computational resources and time.
• Using tensorized representations and CUDA Graph, CoMERA reduced peak memory consumption by 7x, enabling training on smaller GPUs.
• CoMERA’s approach supports diverse architectures, including transformers and large language models, while maintaining or improving accuracy.
• By lowering the energy and resource demands of training, CoMERA contributes to more sustainable AI practices and makes cutting-edge models accessible to a broader audience.

In conclusion, CoMERA addresses some of the most significant barriers to AI scalability and accessibility by enabling faster, memory-efficient training. Its adaptive optimization capabilities and compatibility with modern hardware make it a compelling choice for organizations seeking to train large models without incurring prohibitive costs. This study’s results pave the way for further exploration of tensor-based optimizations in domains like distributed computing and resource-constrained edge devices.

Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. Don’t Forget to join our 60k+ ML SubReddit.

Trending: LG AI Research Releases EXAONE 3.5: Three Open-Source Bilingual Frontier AI-level Models Delivering Unmatched Instruction Following and Long Context Understanding for Global Leadership in Generative AI Excellence….

The post Meet CoMERA: An Advanced Tensor Compression Framework Redefining AI Model Training with Speed and Precision appeared first on MarkTechPost.

Multimodal reasoning—the ability to process and integrate information from diverse data sources such as text, images, and video—remains a demanding area of research in artificial intelligence (AI). Despite advancements, many models still struggle with contextually accurate and efficient cross-modal understanding. These challenges often stem from limitations in scale, narrowly focused datasets, and restricted access to advanced models. Proprietary systems, in particular, can hinder collaborative progress, leaving a gap in the development of more versatile and inclusive AI systems. The need for accessible, high-performing tools is clear as the field works toward practical, generalizable solutions.

The Qwen Team has addressed these challenges by releasing QvQ, an open-weight model specifically designed for multimodal reasoning. Building on the foundation of Qwen2-VL-72B, QvQ integrates architectural improvements that enhance cross-modal reasoning. Its open-weight design underscores the team’s commitment to making advanced AI more accessible.

Technical Innovations and Benefits

QvQ’s architecture is tailored to handle complex multimodal reasoning tasks with efficiency and precision. It employs a hierarchical structure that integrates visual and linguistic information while preserving contextual nuances. This design ensures that computational resources are used effectively without sacrificing accuracy. Additionally, QvQ’s alignment mechanism for text and visual inputs is based on advanced transformer architectures, enabling highly accurate cross-modal embeddings.

With 72 billion parameters, QvQ is built for scalability, capable of handling large and diverse datasets. The open-weight nature of the model allows researchers to customize it for specific applications across domains such as healthcare, education, and creative industries. This flexibility makes QvQ a valuable resource for addressing domain-specific challenges with precision.

Results and Insights

Preliminary evaluations show that QvQ delivers strong performance across key benchmarks in multimodal reasoning. The model has achieved notable results on datasets like Visual7W and VQA, demonstrating its ability to process and respond to complex visual queries with accuracy. These outcomes highlight how QvQ builds on the strengths of Qwen2-VL-72B while incorporating meaningful enhancements.

One of QvQ’s key strengths is its generalization ability. Unlike models that require significant fine-tuning for each new task, QvQ performs effectively across diverse scenarios with minimal adjustment. Its pre-trained architecture, combined with evaluations on cross-domain datasets, underscores its adaptability and potential as a universal tool for multimodal reasoning.

Conclusion

The release of QvQ is a notable step forward in developing advanced multimodal AI systems. By addressing critical challenges and offering a scalable, open-weight solution, the Qwen Team provides a resource that fosters collaboration and innovation. QvQ’s combination of robust technical features and accessibility positions it as a valuable tool for researchers and practitioners. As its applications are explored further, QvQ has the potential to make significant contributions across various fields, advancing the capabilities of AI in multimodal reasoning and beyond.

Check out the demo, model, and details. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. Don’t Forget to join our 60k+ ML SubReddit.

Trending: LG AI Research Releases EXAONE 3.5: Three Open-Source Bilingual Frontier AI-level Models Delivering Unmatched Instruction Following and Long Context Understanding for Global Leadership in Generative AI Excellence….

The post Qwen Team Releases QvQ: An Open-Weight Model for Multimodal Reasoning appeared first on MarkTechPost.

Graphical User Interfaces (GUIs) play a fundamental role in human-computer interaction, providing the medium through which users accomplish tasks across web, desktop, and mobile platforms. Automation in this field is transformative, potentially drastically improving productivity and enabling seamless task execution without requiring manual intervention. Autonomous agents capable of understanding and interacting with GUIs could revolutionize workflows, particularly in repetitive or complex task settings. However, GUIs’ inherent complexity and variability across platforms pose significant challenges. Each platform uses distinct visual layouts, action spaces, and interaction logic, making creating scalable and robust solutions difficult. Developing systems that can navigate these environments autonomously while generalizing across platforms remains an ongoing challenge for researchers in this domain.

There are many technical hurdles in GUI automation right now; one is aligning natural language instructions with the diverse visual representations of GUIs. Traditional methods often rely on textual representations, such as HTML or accessibility trees, to model GUI elements. These approaches are limited because GUIs are inherently visual, and textual abstractions fail to capture the nuances of visual design. In addition, textual representations vary between platforms, leading to fragmented data and inconsistent performance. This mismatch between the visual nature of GUIs and the textual inputs used in automation systems results in reduced scalability, longer inference times, and limited generalization. Also, most current methods are incapable of effective multimodal reasoning and grounding, which are essential for understanding complex visual environments.

Existing tools and techniques have attempted to address these challenges with mixed success. Many systems depend on closed-source models to enhance reasoning and planning capabilities. These models often use natural language communication to combine grounding and reasoning processes, but this approach introduces information loss and lacks scalability. Another common limitation is the fragmented nature of training datasets, which fail to provide comprehensive support for grounding and reasoning tasks. For instance, datasets typically emphasize either grounding or reasoning, but not both, leading to models that excel in one area while struggling in others. This division hampers the development of unified solutions for autonomous GUI interaction.

The University of Hong Kong researchers and Salesforce Research introduced AGUVIS (7B and 72B), a unified framework designed to overcome these limitations by leveraging pure vision-based observations. AGUVIS eliminates the reliance on textual representations and instead focuses on image-based inputs, aligning the model’s structure with the visual nature of GUIs. The framework includes a consistent action space across platforms, facilitating cross-platform generalization. AGUVIS integrates explicit planning and multimodal reasoning to navigate complex digital environments. The researchers constructed a large-scale dataset of GUI agent trajectories, which was used to train AGUVIS in a two-stage process. The framework’s modular architecture, which includes a pluggable action system, allows for seamless adaptation to new environments and tasks.

The AGUVIS framework employs a two-stage training paradigm to equip the model with grounding and reasoning capabilities: 

1. During the first stage, the model focuses on grounding and mapping natural language instructions to visual elements within GUI environments. This stage utilizes a grounding packing strategy, bundling multiple instruction-action pairs into a single GUI screenshot. This method improves training efficiency by maximizing the utility of each image without sacrificing accuracy. 
2. The second stage introduces planning and reasoning, training the model to execute multi-step tasks across various platforms and scenarios. This stage incorporates detailed inner monologues, which include observation descriptions, thoughts, and low-level action instructions. By progressively increasing the complexity of training data, the model learns to handle nuanced tasks with precision and adaptability.

AGUVIS demonstrated great results in both offline and real-world online evaluations. In GUI grounding, the model achieved an average accuracy of 89.2, surpassing state-of-the-art methods across mobile, desktop, and web platforms. In online scenarios, AGUVIS outperformed competing models with a 51.9% improvement in step success rate during offline planning tasks. Also, the model achieved a 93% reduction in inference costs compared to GPT-4o. By focusing on visual observations and integrating a unified action space, AGUVIS sets a new benchmark for GUI automation, making it the first fully autonomous pure vision-based agent capable of completing real-world tasks without reliance on closed-source models.

Key takeaways from the research on AGUVIS in the field of GUI automation:

1. AGUVIS uses image-based inputs, reducing token costs significantly and aligning the model with the inherently visual nature of GUIs. This approach results in a token cost of only 1,200 for 720p image observations, compared to 6,000 for accessibility trees and 4,000 for HTML-based observations.
2. The model combines grounding and planning stages, enabling it to perform single- and multi-step tasks effectively. The grounding training alone equips the model to process multiple instructions within a single image, while the reasoning stage enhances its ability to execute complex workflows.
3. The AGUVIS Collection unifies and augments existing datasets with synthetic data to support multimodal reasoning and grounding. This results in a diverse and scalable dataset, enabling the training of robust and adaptable models.
4. Using pyautogui commands and a pluggable action system allows the model to generalize across platforms while accommodating platform-specific actions, such as swiping on mobile devices.
5. AGUVIS achieved remarkable results in GUI grounding benchmarks, with accuracy rates of 88.3% on web platforms, 85.7% on mobile, and 81.8% on desktops. Also, it demonstrated superior efficiency, reducing USD inference costs by 93% compared to existing models.

In conclusion, the AGUVIS framework addresses critical challenges in grounding, reasoning, and generalization in GUI automation. Its purely vision-based approach eliminates the inefficiencies associated with textual representations, while its unified action space enables seamless interaction across diverse platforms. The research provides a robust solution for autonomous GUI tasks, with applications ranging from productivity tools to advanced AI systems.

Check out the Paper, GitHub Page, and Project. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. Don’t Forget to join our 60k+ ML SubReddit.

Trending: LG AI Research Releases EXAONE 3.5: Three Open-Source Bilingual Frontier AI-level Models Delivering Unmatched Instruction Following and Long Context Understanding for Global Leadership in Generative AI Excellence….

The post Salesforce AI Research Introduces AGUVIS: A Unified Pure Vision Framework Transforming Autonomous GUI Interaction Across Platforms appeared first on MarkTechPost.

Large Language Models (LLMs) have demonstrated impressive proficiency in numerous tasks, but their ability to perform multi-step reasoning remains a significant challenge. This limitation becomes particularly evident in complex scenarios such as mathematical problem-solving, embodied agent control, and web navigation. Traditional Reinforcement Learning (RL) methods, like Proximal Policy Optimization (PPO), have been applied to address this issue but often come with high computational and data costs, making them less practical. Likewise, methods such as Direct Preference Optimization (DPO), while effective for aligning models with human preferences, struggle with multi-step reasoning tasks. DPO’s reliance on pairwise preference data and uniform token treatment undermines its capacity to assign credit effectively in situations with sparse rewards. These obstacles highlight the need for more targeted and efficient solutions to enhance LLM reasoning capabilities.

Introducing OREO: Offline Reasoning Optimization

OREO (Offline REasoning Optimization) is an offline RL approach specifically designed to address the shortcomings of existing methods in improving multi-step reasoning for LLMs. Developed collaboratively by researchers from UC San Diego, Tsinghua University, Salesforce Research, and Northwestern University, OREO builds on insights from maximum entropy reinforcement learning. It trains a policy model and a value function concurrently by optimizing the soft Bellman Equation. This methodology removes the dependency on pairwise preference data, making it possible to utilize unpaired datasets with sparse rewards. Furthermore, OREO enables precise credit assignment across reasoning trajectories, which is especially beneficial when success depends on a few critical steps. The framework can also be extended to iterative exploration setups and incorporates a learned value function to enhance inference through tree search during testing.

Technical Details and Benefits

OREO’s core innovation lies in optimizing the soft Bellman Equation to simultaneously train policy and value models. This strategy ensures accurate credit assignment across reasoning steps, addressing the limitations of methods like DPO. Additionally, OREO offers step-level and response-level objectives, providing flexibility for different granularities of reasoning tasks. During test-time inference, the value function supports advanced search techniques, such as beam search, improving accuracy. Unlike baseline methods like supervised fine-tuning (SFT) or rejection sampling, OREO excels at leveraging failed trajectories to enhance model robustness and adaptability. This capacity to learn from failures makes it particularly valuable for iterative multi-step reasoning tasks.

Results and Insights

OREO’s performance has been rigorously evaluated on benchmarks such as GSM8K and MATH for mathematical reasoning, and ALFWorld for embodied agent control. Key findings include:

• On GSM8K, OREO delivered a 5.2% relative improvement in accuracy using a 1.5B parameter model compared to SFT, alongside a 10.5% improvement on MATH.
• 52.5% on MATH with 1.5B LLM (w/o using augmented problem set)
• In ALFWorld, OREO achieved a 17.7% relative improvement in performance in unseen environments, underscoring its ability to generalize beyond training data.

Iterative training further amplified OREO’s effectiveness, showing consistent accuracy gains over multiple iterations. While approaches like rejection sampling exhibited diminishing returns, OREO continued to improve by incorporating insights from failed attempts. Test-time search using OREO’s value function resulted in up to a 17.9% relative improvement over greedy decoding on the MATH dataset, highlighting its impact on inference quality.

Conclusion

OREO provides a practical and effective solution for enhancing multi-step reasoning in LLMs through offline RL. By addressing the limitations of existing approaches, it offers a scalable method for improving reasoning capabilities. Its integration of detailed credit assignment, iterative training, and test-time search makes it a versatile tool for addressing complex reasoning challenges. The results demonstrate OREO’s potential for application across a range of domains requiring sophisticated problem-solving, contributing to the evolution of AI systems capable of deeper reasoning.

Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. Don’t Forget to join our 60k+ ML SubReddit.

Trending: LG AI Research Releases EXAONE 3.5: Three Open-Source Bilingual Frontier AI-level Models Delivering Unmatched Instruction Following and Long Context Understanding for Global Leadership in Generative AI Excellence….

The post Meet OREO (Offline REasoning Optimization): An Offline Reinforcement Learning Method for Enhancing LLM Multi-Step Reasoning appeared first on MarkTechPost.

The widespread use of large-scale language models (LLMs) in safety-critical areas has brought forward a crucial challenge: how to ensure their adherence to clear ethical and safety guidelines. Existing alignment techniques, such as supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF), have limitations. Models can still produce harmful content when manipulated, refuse legitimate requests, or struggle to handle unfamiliar scenarios. These issues often stem from the implicit nature of current safety training, where models infer standards indirectly from data rather than learning them explicitly. Additionally, models generally lack the ability to deliberate on complex prompts, which limits their effectiveness in nuanced or adversarial situations.

OpenAI researchers have introduced Deliberative Alignment, a new approach that directly teaches models safety specifications and trains them to reason over these guidelines before generating responses. By integrating safety principles into the reasoning process, this method addresses key weaknesses in traditional alignment techniques. Deliberative Alignment focuses on teaching models to explicitly consider relevant policies, enabling them to handle complex scenarios more reliably. Unlike approaches that depend heavily on human-annotated data, this method uses model-generated data and chain-of-thought (CoT) reasoning to achieve better safety outcomes. When applied to OpenAI’s o-series models, it has demonstrated improved resistance to jailbreak attacks, fewer refusals of valid requests, and better generalization to unfamiliar situations.

Technical Details and Benefits

Deliberative Alignment involves a two-stage training process. First, supervised fine-tuning (SFT) trains models to reference and reason through safety specifications using datasets generated from base models. This step helps embed a clear understanding of safety principles. In the second stage, reinforcement learning (RL) refines the model’s reasoning using a reward model to evaluate performance against safety benchmarks. This training pipeline does not rely on human-annotated completions, which reduces the resource demands typically associated with safety training. By leveraging synthetic data and CoT reasoning, Deliberative Alignment equips models to address complex ethical scenarios with greater precision and efficiency.

Results and Insights

Deliberative Alignment has yielded notable improvements in the performance of OpenAI’s o-series models. The o1 model, for instance, outperformed other leading models in resisting jailbreak prompts, achieving a 0.88 score on the StrongREJECT benchmark compared to GPT-4o’s 0.37. It also performed well in avoiding unnecessary refusals, with a 93% accuracy rate on benign prompts in the XSTest dataset. The method further improved adherence to style guidelines in responses to regulated advice and self-harm prompts. Ablation studies have shown that both SFT and RL stages are essential for achieving these results. Additionally, the approach has demonstrated strong generalization to out-of-distribution scenarios, such as multilingual and encoded prompts, highlighting its robustness.

Conclusion

Deliberative Alignment represents a significant advancement in aligning language models with safety principles. By teaching models to reason explicitly over safety policies, it offers a scalable and interpretable solution to complex ethical challenges. The success of the o1 series models illustrates the potential of this approach to improve safety and reliability in AI systems. As the capabilities of AI continue to evolve, methods like Deliberative Alignment will play a crucial role in ensuring that these systems remain aligned with human values and expectations.

Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. Don’t Forget to join our 60k+ ML SubReddit.

Trending: LG AI Research Releases EXAONE 3.5: Three Open-Source Bilingual Frontier AI-level Models Delivering Unmatched Instruction Following and Long Context Understanding for Global Leadership in Generative AI Excellence….

The post OpenAI Researchers Propose ‘Deliberative Alignment’: A Training Approach that Teaches LLMs to Explicitly Reason through Safety Specifications before Producing an Answer appeared first on MarkTechPost.