Sana Hassan, Author at MarkTechPost

Camel-AI Open Sourced OASIS: A Next Generation Simulator for Realistic Social Media Dynamics with One Million Agents

Sana Hassan — Sat, 28 Dec 2024 07:38:21 +0000

Social media platforms have revolutionized human interaction, creating dynamic environments where millions of users exchange information, form communities, and influence one another. These platforms, including X and Reddit, are not just tools for communication but have become critical ecosystems for understanding modern societal behaviors. Simulating such intricate interactions is vital for studying misinformation, group polarization, and herd behavior. Computational models provide researchers a cost-effective and scalable way to analyze these interactions without conducting resource-intensive real-world experiments. But, creating models replicating the scale and complexity of social networks remains a significant challenge.

The primary issue in modeling social media is capturing millions of users’ diverse behaviors and interactions in a dynamic network. Traditional agent-based models (ABMs) fall short of representing complex behaviors like context-driven decision-making or the influence of dynamic recommendation algorithms. Also, existing models are often limited to small-scale simulations, typically involving only hundreds or thousands of agents, which restricts their ability to mimic large-scale social systems. Such constraints hinder researchers from fully exploring phenomena like how misinformation spreads or how group dynamics evolve in online environments. These limitations highlight the need for more advanced and scalable simulation tools.

Existing methods for simulating social media interactions often lack essential features like dynamic user networks, detailed recommendation systems, and real-time updates. For instance, most ABMs rely on pre-programmed agent behaviors, which fail to reflect the nuanced decision-making seen in real-world users. Also, current simulators are typically platform-specific, designed to study isolated phenomena, making them impractical for broader applications. They cannot often scale beyond a few thousand agents, leaving researchers unable to examine the behaviors of millions of users interacting simultaneously. The absence of scalable, versatile models has been a major bottleneck in advancing social media research.

Researchers from Camel-AI, Shanghai Artificial Intelligence Laboratory, Dalian University of Technology, Oxford, KAUST, Fudan University, Xi’an Jiaotong University, Imperial College London, Max Planck Institute, and The University of Sydney developed OASIS, a next-generation social media simulator designed for scalability and adaptability to address these challenges. OASIS is built upon modular components, including an Environment Server, Recommendation System (RecSys), Time Engine, and Agent Module. It supports up to one million agents, making it one of the most comprehensive simulators. This system incorporates dynamically updated networks, diverse action spaces, and advanced algorithms to replicate real-world social media dynamics. By integrating data-driven methods and open-source frameworks, OASIS provides a flexible platform for studying phenomena across platforms like X and Reddit, enabling researchers to explore topics ranging from information propagation to herd behavior.

The architecture of OASIS emphasizes both scale and functionality. The functions of some of the components are as follows:

Its Environment Server is the backbone, storing detailed user profiles, historical interactions, and social connections.
The Recommendation System customizes content visibility using advanced algorithms such as TwHIN-BERT, which processes user interests and recent activities to rank posts.
The Time Engine governs user activation based on hourly probabilities, simulating realistic online behavior patterns.

These components work together to create a simulation environment that can adapt to different platforms and scenarios. Switching from X to Reddit requires minimal module adjustments, making OASIS a versatile tool for social media research. Its distributed computing infrastructure ensures efficient handling of large-scale simulations, even with up to one million agents.

In experiments modeling information propagation on X, OASIS achieved a normalized RMSE of approximately 30%, demonstrating its ability to align with actual dissemination trends. The simulator also replicated group polarization, showing that agents tend to adopt more extreme opinions during interactions. This effect was particularly pronounced in uncensored models, where agents used more extreme language. Moreover, OASIS revealed unique insights, such as the herd effect being more evident in agents than in humans. Agents consistently followed negative trends when exposed to down-treated comments, while humans displayed a stronger critical approach. These findings underscore the simulator’s potential to uncover both expected and novel patterns in social behavior.

With OASIS, larger agent groups lead to richer and more diverse interactions. For example, when the number of agents increased from 196 to 10,196, the diversity and helpfulness of user responses improved significantly, with a 76.5% increase in perceived helpfulness. At an even larger scale of 100,196 agents, user interactions became more varied and meaningful, illustrating the importance of scalability in studying group behavior. Also, OASIS demonstrated that misinformation spreads more effectively than truthful information, particularly when rumors are emotionally provocative. The simulator also showed how isolated user groups form over time, providing valuable insights into the dynamics of online communities.

Key takeaways from the OASIS research include:

OASIS can simulate up to one million agents, far surpassing the capabilities of existing models.
It supports multiple platforms, including X and Reddit, with modular components that are easily adjustable.
The simulator replicates phenomena like group polarization and herd behavior, providing a deeper understanding of these dynamics.
OASIS achieved a normalized RMSE of 30% in information propagation experiments, closely aligning with real-world trends.
It demonstrated that rumors spread faster and more widely than truthful information in large-scale simulations.
Larger agent groups enhance the diversity and helpfulness of responses, emphasizing the importance of scale in social media studies.
OASIS distributed computing allows for efficient handling of simulations, even with millions of agents.

In conclusion, OASIS is a breakthrough in simulating social media dynamics, offering scalability and adaptability. OASIS addresses existing model limitations and provides a robust framework for studying complex-scale interactions. Integrating LLMs with rule-based agents accurately mimics the behaviors of up to one million users across platforms like X and Reddit. Its ability to replicate complex phenomena, such as information propagation, group polarization, and herd effects, provides researchers valuable insights into modern social ecosystems.

Check out the Paper and GitHub Page. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. Don’t Forget to join our 60k+ ML SubReddit.

The post Camel-AI Open Sourced OASIS: A Next Generation Simulator for Realistic Social Media Dynamics with One Million Agents appeared first on MarkTechPost.

Unveiling Privacy Risks in Machine Unlearning: Reconstruction Attacks on Deleted Data

Sana Hassan — Sat, 28 Dec 2024 01:42:21 +0000

Machine unlearning is driven by the need for data autonomy, allowing individuals to request the removal of their data’s influence on machine learning models. This field complements data privacy efforts, which focus on preventing models from revealing sensitive information about the training data through attacks like membership inference or reconstruction. While differential privacy methods limit these risks, unlearning enables the deletion of data from a trained model, ensuring it behaves as if the data were never included in the first place. Achieving this efficiently, without retraining the entire model, has been a key focus, particularly for complex models like deep neural networks.

However, unlearning introduces new privacy risks. When adversaries compare a model’s parameters before and after data deletion, they can exploit the differences to reconstruct the deleted data, even for simple models like linear regression. This process leverages the gradient of the deleted sample and the expected Hessian derived from public data to approximate the changes caused by unlearning. The approach highlights a unique vulnerability where unlearning unintentionally exposes sensitive data. By extending existing techniques for gradient-based reconstruction attacks, this research reveals how unlearning can facilitate exact data reconstruction, emphasizing the importance of safeguards like differential privacy to mitigate these risks.

Researchers from AWS AI, the University of Pennsylvania, the University of Washington, Carnegie Mellon University, and Jump Trading reveal that data deletion in machine learning models, even simple ones, exposes individuals to high-accuracy reconstruction attacks. These attacks recover deleted data by exploiting differences in model parameters before and after deletion. The study demonstrates effective attacks on linear regression models using closed-form training algorithms and extends these methods to models with pre-trained embeddings and generic architectures via Newton’s method. Experiments on tabular and image datasets highlight significant privacy risks in retraining for unlearning without safeguards like differential privacy.

The researchers present an attack to reconstruct deleted user data from regularized linear regression models by analyzing parameter changes before and after deletion. The method leverages the relationship between model parameters and the removed sample, approximating key statistics using public data. The approach generalizes to models with fixed embeddings and extends to non-linear architectures using Newton’s approximation method. Experiments demonstrate its applicability to multiclass classification and label inference by estimating gradients and reconstructing deleted data. This highlights the vulnerability of models to privacy breaches, especially without safeguards, as the attack remains effective across various architectures and loss functions.

The study evaluates our attack across diverse datasets for classification and regression tasks, including tabular and image data. Using full retraining, they compare model parameters before and after a single sample’s deletion. Our method leverages public data from the same distribution without needing knowledge of the deleted sample. Against baselines like “Avg” (average of public samples) and “MaxDiff” (maximizing parameter change), our attack consistently outperforms, achieving higher cosine similarity with deleted samples. Tested on MNIST, CIFAR10, and ACS income data, our approach reconstructs deleted samples effectively across various models, emphasizing vulnerabilities in machine learning systems and the need for privacy safeguards.

In conclusion, The work introduces a reconstruction attack capable of recovering deleted data from simple machine-learning models with high accuracy. The attack achieves near-perfect results for linear regression and performs effectively on models using embeddings or optimizing different loss functions. Highlighting privacy risks in data deletion or machine unlearning, the findings emphasize the need for techniques like differential privacy. Counterintuitively, data deletion updates can increase vulnerability to reconstruction attacks, even in basic models, exposing sensitive data. Through extensive experiments on diverse datasets, this study underscores the significant privacy risks posed by data deletion requests, even in seemingly low-risk model settings.

Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. Don’t Forget to join our 60k+ ML SubReddit.

The post Unveiling Privacy Risks in Machine Unlearning: Reconstruction Attacks on Deleted Data appeared first on MarkTechPost.

Neural Networks for Scalable Temporal Logic Model Checking in Hardware Verification

Sana Hassan — Thu, 26 Dec 2024 16:23:27 +0000

Ensuring the correctness of electronic designs is critical, as hardware flaws are permanent post-production and can compromise software reliability or the safety of cyber-physical systems. Verification is central to digital circuit engineering, with FPGA and IC/ASIC projects dedicating 40% and 60% of their time, respectively, to this process. While testing approaches, such as directed or constrained random testing, are easy to implement, they are inherently non-exhaustive and cannot ensure the absence of critical errors. Formal verification, particularly model checking, addresses these limitations by mathematically confirming whether a design satisfies its specifications across all possible executions. However, methods like BDDs and SAT solvers remain computationally intensive and struggle to scale for complex circuits. Engineers often rely on bounded model checking to reduce computational demands, which sacrifices global correctness over extended time horizons.

Formal verification has evolved over decades, with temporal logic playing a key role in describing system behaviors. Based on Linear Temporal Logic (LTL), SystemVerilog Assertions are widely used to define safety and liveness properties. Safety properties are efficiently verified using BDDs, while SAT-based methods scale better for bounded model checking but remain incomplete without achieving impractically high thresholds. Advanced techniques like IC3 and Craig Interpolation improve unbounded safety checking, while Emerson-Lei fixed-point computations and k-liveness extend verification to liveness properties. Verifying systems with complex arithmetic remains challenging, often requiring explicit-state abstractions, inductive invariants, or ranking functions. Originally developed for software termination analysis, ranking functions have been generalized for hardware liveness verification, incorporating non-linear, piecewise-defined, and lexicographic methods to address modern system complexities.

Researchers from the University of Birmingham, Amazon Web Services, and Queen Mary University of London have developed a machine learning-based approach for hardware model checking that integrates neural networks and symbolic reasoning. Their method uses neural networks to represent proof certificates for LTL specifications, trained from randomly generated system executions. The approach guarantees formal correctness over unbounded time horizons by employing satisfiability solving to validate these certificates. Experiments demonstrate its effectiveness, outperforming both academic and commercial model checkers in speed and task completion across standard hardware verification problems, contributing to improved safety and reliability in system designs.

LTL model checking verifies if all possible sequences of actions in a system (M) comply with a given LTL formula (Phi), which describes the desired temporal properties. The system (M) includes input and state variables, with its behavior determined by transition rules. To check this, (Phi) is converted into a type of automaton called a Büchi automaton (A_Phi). The verification ensures that the combined system (M) and the automaton (A_neg Phi) (representing the formula’s negation) have no valid infinite sequences. Neural ranking functions aid in proving termination and are validated using SMT solvers.

The experimental evaluation tested 194 verification tasks derived from 10 parameterized hardware designs with varying complexity. A prototype neural model-checking tool was developed, using Spot to generate automata, Verilator for data generation, PyTorch for training, and Bitwuzla for SMT-solving. The tool was benchmarked against industry leaders ABC, nuXmv, and anonymized tools X and Y. It completed 93% of tasks, outperforming competitors in scalability and runtime, although challenges like local minima and extended SMT-check times remain. While generally faster, it struggled with trivial tasks like UARTt due to overhead. The method’s limitations include reliance on word-level inputs and risks of dataset bias.

In conclusion, the study introduces an approach to model-checking temporal logic using neural networks as proof certificates for hardware verification. Neural networks are trained on synthetic system executions, leveraging their ability to represent ranking functions for fair termination. The method combines machine learning and symbolic reasoning by validating neural certificates with satisfiability solvers, ensuring formal guarantees. Applied to SystemVerilog designs, it outperforms state-of-the-art tools in scalability. Despite the computational demand of SMT solving, the approach is effective with simple feed-forward networks. This marks the first successful use of neural certificates for temporal logic, establishing a foundation for further advancements in model checking.

Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. Don’t Forget to join our 60k+ ML SubReddit.

The post Neural Networks for Scalable Temporal Logic Model Checking in Hardware Verification appeared first on MarkTechPost.

Tencent Research Introduces DRT-o1: Two Variants DRT-o1-7B and DRT-o1-14B with Breakthrough in Neural Machine Translation for Literary Texts

Sana Hassan — Wed, 25 Dec 2024 21:05:57 +0000

Neural machine translation (NMT) is a sophisticated branch of natural language processing that automates text conversion between languages using machine learning models. Over the years, it has become an indispensable tool for global communication, with applications spanning diverse areas such as technical document translation and digital content localization. Despite its advancements in translating straightforward text, NMT faces persistent challenges in handling literary content rich in metaphors and similes. These expressions carry deep cultural and contextual nuances, making their translation far more complex. Conventional systems often resort to literal translations, which can fail to preserve the intended meaning and cultural essence, particularly in literature, where semantics are intertwined with artistic and emotional undertones.

Translating idiomatic expressions and metaphorical content involves unique difficulties stemming from their reliance on cultural interpretation. Literal translations of such constructs often lead to a loss of nuance, rendering the output confusing or meaningless to native speakers. This issue persists even with the most advanced NMT systems, designed to excel in tasks involving structured or technical text but falter when interpreting abstract and figurative language. Human translators invest considerable effort in reinterpreting these expressions to ensure they align with the target audience’s cultural framework while retaining the original intent. Bridging this gap in automated systems requires a novel approach capable of mimicking this human adaptability.

Existing NMT tools leverage supervised fine-tuning (SFT) techniques to enhance translation capabilities. These tools typically rely on datasets optimized for technical or straightforward text, such as manuals or academic papers. However, their performance diminishes when dealing with metaphorical or idiomatic language. Systems like Qwen2.5 and Marco-O1 improve accuracy and fluency for basic translations but remain ill-equipped to handle the layered complexities of literary language. For instance, Qwen2.5-7B achieves a BLEU score of 27.02, and Qwen2.5-14B improves this to 30.23, yet neither comes close to meeting the high expectations of literary translation where context and nuance are paramount.

Researchers from Tencent Inc. have developed an innovative system called DRT-o1 to overcome these limitations. It comprises of two variants:

They are built upon the Qwen2.5 backbones and integrate a novel multi-agent framework to address the intricacies of metaphorical and idiomatic translation. The researchers focused on literature as their primary domain, mining approximately 400 public-domain English books from Project Gutenberg. They extracted 577,600 sentences and filtered them to retain only 63,000 containing similes and metaphors. These sentences were deemed suitable for what the researchers describe as “long thought” processes in machine translation. Unlike previous approaches, the DRT-o1 system relies on a collaborative method involving three agents:

A translator
An advisor
An evaluator

Each agent iteratively refines the translation, ensuring that every output improves upon the last.

The multi-agent framework at the core of DRT-o1 begins with identifying key terms in a source sentence. These terms are translated individually to ensure contextual accuracy. The framework then generates a preliminary translation, which undergoes multiple refinement loops. During each iteration, the advisor provides feedback on the current translation, and the evaluator assigns a score based on predefined quality metrics. This iterative process continues until the evaluator’s score meets a predefined threshold or the maximum number of iterations is reached. The outputs are then polished for fluency and readability using GPT-4o, creating a final dataset of 22,264 long-thought machine translation samples.

The DRT-o1 system and its variants significantly improve performance over existing NMT models. Experimental results reveal that DRT-o1-7B achieves an 8.26-point increase in BLEU score and a 3.36-point rise in CometScore compared to its Qwen2.5-7B-Instruct counterpart. Similarly, DRT-o1-14B records a BLEU improvement of 7.33 and a CometScore increase of 1.66 over Qwen2.5-14B-Instruct. These results underscore the effectiveness of the multi-agent framework in capturing the subtleties of literary translation. Notably, DRT-o1-7B even outperforms larger models such as QwQ-32B, demonstrating the scalability and efficiency of this system. For example, the 7B variant surpasses QwQ-32B by 7.82 BLEU points and 1.46 CometScore, further establishing its capabilities in handling complex linguistic constructs.

Key takeaways from the research on the DRT-o1:

The dataset creation involved mining 577,600 sentences from 400 public-domain books, filtering them to 63,000 suitable for long-thought processes.
The multi-agent framework employs three roles – translator, advisor, and evaluator – to iteratively refine translations and ensure superior output quality.
DRT-o1-7B improved its BLEU by 8.26 points, while DRT-o1-14B recorded a 7.33-point increase, showcasing the system’s ability to outperform existing models.
The integration of GPT-4o ensures fluency and readability, further enhancing the quality of machine translations.
DRT-o1-7B outperformed the larger QwQ-32B model by 7.82 BLEU points, highlighting its scalability and efficiency in translating complex literary content.

In conclusion, the DRT-o1 system and its variants (DRT-o1-7B and DRT-o1-14B) represent a transformative approach to neural machine translation. The researchers have addressed long-standing challenges by focusing on literary language and integrating a sophisticated multi-agent framework. The iterative refinement process preserves the meaning and cultural context of metaphors and similes and achieves performance metrics that surpass state-of-the-art models. This work underscores the potential of long-chain reasoning in enhancing NMT, providing a scalable and effective solution for translating nuanced text with precision and cultural sensitivity.

The post Tencent Research Introduces DRT-o1: Two Variants DRT-o1-7B and DRT-o1-14B with Breakthrough in Neural Machine Translation for Literary Texts appeared first on MarkTechPost.

This AI Paper by The Data Provenance Initiative Team Highlights Challenges in Multimodal Dataset Provenance, Licensing, Representation, and Transparency for Responsible Development

Sana Hassan — Wed, 25 Dec 2024 01:31:25 +0000

The advancement of artificial intelligence hinges on the availability and quality of training data, particularly as multimodal foundation models grow in prominence. These models rely on diverse datasets spanning text, speech, and video to enable language processing, speech recognition, and video content generation tasks. However, the lack of transparency regarding dataset origins and attributes creates significant barriers. Using training data that is geographically and linguistically skewed, inconsistently licensed, or poorly documented introduces ethical, legal, and technical challenges. Understanding the gaps in data provenance is essential for advancing responsible and inclusive AI technologies.

AI systems face a critical issue in dataset representation and traceability, which limits the development of unbiased and legally sound technologies. Current datasets often rely heavily on a few web-based or synthetically generated sources. These include platforms like YouTube, which accounts for a significant share of speech and video datasets, and Wikipedia, which dominates text data. This dependency results in datasets failing to represent underrepresented languages and regions adequately. In addition, the unclear licensing practices of many datasets create legal ambiguities, as more than 80% of widely used datasets carry some form of undocumented or implicit restrictions despite only 33% being explicitly licensed for non-commercial use.

Attempts to address these challenges have traditionally focused on narrow aspects of data curation, such as removing harmful content or mitigating bias in text datasets. However, such efforts are typically limited to single modalities and lack a comprehensive framework to evaluate datasets across modalities like speech and video. Platforms hosting these datasets, such as HuggingFace or OpenSLR, often lack the mechanisms to ensure metadata accuracy or enforce consistent documentation practices. This fragmented approach underscores the urgent need for a systematic audit of multimodal datasets that holistically considers their sourcing, licensing, and representation.

To close this gap, researchers from the Data Provenance Initiative conducted the largest longitudinal audit of multimodal datasets, examining nearly 4,000 public datasets created between 1990 and 2024. The audit spanned 659 organizations from 67 countries, covering 608 languages and nearly 1.9 million hours of speech and video data. This extensive analysis revealed that web-crawled and social media platforms now account for most training data, with synthetic sources also rapidly growing. The study highlighted that while only 25% of text datasets have explicitly restrictive licenses, nearly all content sourced from platforms like YouTube or OpenAI carries implicit non-commercial constraints, raising questions about legal compliance and ethical use.

The researchers applied a meticulous methodology to annotate datasets, tracing their lineage back to sources. This process uncovered significant inconsistencies in how data is licensed and documented. For instance, while 96% of text datasets include commercial licenses, over 80% of their source materials impose restrictions that are not carried forward in the dataset’s documentation. Similarly, video datasets highly depended on proprietary or restricted platforms, with 71% of video data originating from YouTube alone. Such findings underscore the challenges practitioners face in accessing data responsibly, particularly when datasets are repackaged or re-licensed without preserving their original terms.

Notable findings from the audit include the dominance of web-sourced data, particularly for speech and video. YouTube emerged as the most significant source, contributing nearly 1 million hours to each speech and video content, surpassing other sources like audiobooks or movies. Synthetic datasets, while still a smaller portion of overall data, have grown rapidly, with models like GPT-4 contributing significantly. The audit also revealed stark geographical imbalances. North American and European organizations accounted for 93% of text data, 61% of speech data, and 60% of video data. In comparison, regions like Africa and South America collectively represented less than 0.2% across all modalities.

Geographical and linguistic representation remains a persistent challenge despite nominal increases in diversity. Over the past decade, the number of languages represented in training datasets has grown to over 600, yet measures of equality in representation have shown no significant improvement. The Gini coefficient, which measures inequality, remains above 0.7 for geographical distribution and above 0.8 for language representation in text datasets, highlighting the disproportionate concentration of contributions from Western countries. For speech datasets, while representation from Asian countries like China and India has improved, African and South American organizations continue to lag far behind.

The research provides several critical takeaways, offering valuable insights for developers and policymakers:

Over 70% of speech and video datasets are derived from web platforms like YouTube, while synthetic sources are becoming increasingly popular, accounting for nearly 10% of all text data tokens.
While only 33% of datasets are explicitly non-commercial, over 80% of source content is restricted. This mismatch complicates legal compliance and ethical use.
North American and European organizations dominate dataset creation, with African and South American contributions at less than 0.2%. Linguistic diversity has grown nominally but remains concentrated in many dominant languages.
GPT-4, ChatGPT, and other models have significantly contributed to the rise of synthetic datasets, which now represent a growing share of training data, particularly for creative and generative tasks.
The lack of transparency and persistent Western-centric biases call for more rigorous audits and equitable practices in dataset curation.

In conclusion, this comprehensive audit sheds light on the growing reliance on web-crawled and synthetic data, the persistent inequalities in representation, and the complexities of licensing in multimodal datasets. By identifying these challenges, the researchers provide a roadmap for creating more transparent, equitable, and responsible AI systems. Their work underscores the need for continued vigilance and measures to ensure that AI serves diverse communities fairly and effectively. This study is a call to action for practitioners, policymakers, and researchers to address the structural inequities in the AI data ecosystem and prioritize transparency in data provenance.

The post This AI Paper by The Data Provenance Initiative Team Highlights Challenges in Multimodal Dataset Provenance, Licensing, Representation, and Transparency for Responsible Development appeared first on MarkTechPost.

Redesigning Datasets for AI-Driven Mathematical Discovery: Overcoming Current Limitations and Enhancing Workflow Representation

Sana Hassan — Tue, 24 Dec 2024 20:33:46 +0000

Current datasets used to train and evaluate AI-based mathematical assistants, particularly LLMs, are limited in scope and design. They often focus on undergraduate-level mathematics and rely on binary rating protocols, making them unsuitable for evaluating complex proof-based reasoning comprehensively. These datasets lack representation of critical aspects of mathematical workflows, such as intermediate steps and problem-solving strategies essential in mathematical research. To overcome these limitations, there is a pressing need to redesign datasets to include elements like “motivated proofs,” which emphasize reasoning processes over results, and workflows that capture the nuanced tasks involved in mathematical discovery.

Recent advancements in AI for mathematics, such as AlphaGeometry and Numina, have successfully solved Olympiad-level problems and converted mathematical queries into executable code. However, the proliferation of benchmarks, such as GSM8K and MATH, has led to over-reliance on a few datasets while neglecting advanced mathematics and practical workflows. While highly specialized models excel in narrow domains requiring formal language input, general-purpose models like LLMs aim to assist mathematicians broadly through natural language interaction and tool integration. Despite their progress, these systems face challenges such as dataset contamination and lack of alignment with real-world mathematical practices, highlighting the need for more comprehensive evaluation methods and training data.

Researchers from institutions like Oxford, Cambridge, Caltech, and Meta emphasize improving LLMs to serve as effective “mathematical copilots.” Current datasets, such as GSM8K and MATH, fall short of capturing the nuanced workflows and motivations central to mathematical research. The authors advocate for a shift towards datasets reflecting practical mathematical tasks inspired by concepts like Pólya’s “motivated proof.” They propose integrating symbolic tools and specialized LLM modules to enhance reasoning alongside developing universal models for theorem discovery. The study underscores the importance of datasets tailored to mathematicians’ needs to guide the development of more capable AI systems.

While not specifically designed for mathematics, current general-purpose LLMs have demonstrated strong capabilities in solving complex problems and generating mathematical text. GPT-4, for example, performs well on undergraduate-level math problems, and Google’s Math-Specialized Gemini 1.5 Pro has achieved over 90% accuracy on the MATH dataset. Despite these advancements, concerns exist regarding the reproducibility of results, as datasets may be contaminated or not properly tested, potentially affecting generalization to diverse problem types. Specialized models like MathPrompter and MathVista perform well in arithmetic and geometry but are limited by the narrow focus of available datasets, often omitting advanced reasoning tasks.

The study highlights how current datasets fail to support AI models in addressing the full spectrum of mathematical research, particularly in tasks like conjecture generation and proof strategies. Existing datasets primarily focus on question-answering or theorem proving without evaluating the intermediate reasoning process or workflows mathematicians follow. Many formal datasets lack problem complexity, suffer from tool misalignment, or face data duplication issues. To overcome these challenges, the paper advocates for developing new datasets encompassing a wide range of mathematical research activities, such as literature search and proof formulation, along with a comprehensive taxonomy of workflows to guide future model development.

In conclusion, The study discusses AI’s challenges in becoming a true mathematical partner, similar to GitHub Copilot for programmers. It highlights the complementary nature of natural and formal language datasets, noting that what is easy in one representation may be difficult in the other. The authors emphasize the need for better datasets that capture mathematical workflows, intermediate steps, and the ability to assess proof techniques. They argue for developing datasets beyond proofs and results to include reasoning, heuristics, and summarization, which will aid AI in accelerating mathematical discovery and supporting other scientific disciplines.

The post Redesigning Datasets for AI-Driven Mathematical Discovery: Overcoming Current Limitations and Enhancing Workflow Representation appeared first on MarkTechPost.

Viro3D: A Comprehensive Resource of Predicted Viral Protein Structures Unveils Evolutionary Insights and Functional Annotations

Sana Hassan — Sun, 22 Dec 2024 20:20:29 +0000

Viruses infect organisms across all domains of life, playing key roles in ecological processes such as ocean biogeochemical cycles and the regulation of microbial populations while also causing diseases in humans, animals, and plants. Viruses are Earth’s most abundant biological entities, characterized by rapid evolution, high mutation rates, and frequent genetic exchanges with hosts and other viruses. This constant genetic flux leads to highly diverse genomes with mosaic architectures, challenging functional annotation, evolutionary analysis, and taxonomic classification. Viruses have likely emerged multiple times throughout history despite their diversity, with some lineages predating the last universal common ancestor (LUCA). This highlights a longstanding co-evolutionary relationship between viruses and cellular organisms.

Protein structures, more conserved than sequences, offer a reliable means to study evolutionary relationships and infer gene functions in viruses. However, viral protein structures are significantly underrepresented in public databases, with less than 10% of the Protein Data Bank (PDB) comprising experimental viral protein structures. Recent advances in machine learning, such as AlphaFold2 and ESMFold, have enabled accurate protein structure prediction at scale. Using these tools, researchers have generated a comprehensive dataset of 85,000 predicted structures from 4,400 human and animal viruses, significantly expanding structural coverage. These efforts address the historical gap in viral protein representation, facilitating functional annotation and phylogenetic analysis and shedding light on the evolutionary history of critical viral proteins like class-I fusion glycoproteins.

Researchers from the MRC-University of Glasgow Centre for Virus Research and the University of Tokyo generated 170,000 predicted protein structures from 4,400 animal viruses using ColabFold and ESMFold. They evaluated model quality, performed structural analyses, and explored deep phylogenetic relationships, particularly focusing on class-I membrane fusion glycoproteins, including the origins of coronavirus spike proteins. To support the virology community, they developed Viro3D, an accessible database where users can search, browse, and download viral protein models and explore structural similarities across virus species. This resource aims to advance molecular virology, virus evolution studies, and the design of therapies and vaccines.

The study utilized 6,721 GenBank nucleotide accession numbers, covering 4,407 virus isolates and 3,106 species with host annotations, to extract 71,269 viral protein records. Additional annotations included 4,070 mature peptides, 11,786 protein regions, and 253 polyproteins. Protein structures were predicted using ColabFold and ESMFold, with structural coverage evaluated against the PDB. Proteins were clustered based on sequence and structural similarity, forming 19,067 structural clusters. Functional annotations were expanded using sequence-based and structural networks. A structural similarity map of viral species was created, and comparisons were made with other viral structure databases, highlighting the dataset’s comprehensiveness and structural insights.

The study introduced Viro3D, a robust database encompassing over 170,000 predicted 3D protein structures from 4,400 animal viruses. Using ColabFold and ESMFold, researchers achieved a significant 30-fold increase in structural coverage compared to experimental data. Notably, this dataset revealed functional and evolutionary insights, including the evolutionary origins of coronavirus spike proteins. Structural analyses and protein-protein interaction networks supported functional annotations. Viro3D’s predictions showed high reliability when benchmarked against experimentally solved viral structures. Viro3D provides an unprecedented resource for studying viral evolution, protein function, and structural mechanisms, offering potential applications in antiviral drug and vaccine development.

In conclusion, the study expanded viral protein structural coverage 30-fold by modeling 85,000 proteins from 4,400 human and animal viruses, with 64% of models being highly confident. Combining ColabFold and ESMFold methods enhanced efficiency, accuracy, and speed. Structural clustering reduced viral diversity to 19,000 distinct structures, 65% unique to this dataset, with many found near viral genome ends, suggesting evolutionary hotspots. Analysis revealed that viral proteins often lack homologs in cellular organisms, indicating extensive remodeling. The study traced their evolution by exploring class-I fusion glycoproteins, highlighting their role in virus transmission and pathogenesis, and offering valuable insights for virology research.

The post Viro3D: A Comprehensive Resource of Predicted Viral Protein Structures Unveils Evolutionary Insights and Functional Annotations appeared first on MarkTechPost.

OpenAI Researchers Propose Comprehensive Set of Practices for Enhancing Safety, Accountability, and Efficiency in Agentic AI Systems

Sana Hassan — Sun, 22 Dec 2024 07:45:35 +0000

Agentic AI systems are fundamentally reshaping how tasks are automated, and goals are achieved in various domains. These systems are distinct from conventional AI tools in that they can adaptively pursue complex goals over extended periods with minimal human supervision. Their functionality extends to tasks requiring reasoning, such as managing logistics, developing software, or even handling customer service at scale. The potential for these systems to enhance productivity, reduce human error, and accelerate innovation makes them a focal point for researchers and industry stakeholders. However, these systems’ growing complexity and autonomy necessitate the development of rigorous safety, accountability, and operational frameworks.

Despite their promise, agentic AI systems pose significant challenges that demand attention. Unlike traditional AI, which performs predefined tasks, agentic systems must navigate dynamic environments while aligning with user intentions. This autonomy introduces vulnerabilities, such as the possibility of unintended actions, ethical conflicts, and the risk of exploitation by malicious actors. Also, as these systems are deployed across diverse applications, the stakes rise considerably, particularly in high-impact sectors such as healthcare, finance, and defense. The absence of standardized protocols exacerbates these challenges, as developers and users lack a unified approach to managing potential risks.

While effective in specific contexts, current approaches to AI safety often fall short when applied to agentic systems. For example, rule-based systems and manual oversight mechanisms are ill-suited for environments requiring rapid, autonomous decision-making. Traditional evaluation methods also struggle to capture the intricacies of multi-step, goal-oriented behaviors. Also, techniques such as human-in-the-loop systems, which aim to keep users involved in decision-making, are constrained by scalability issues and can introduce inefficiencies. Existing safeguards also fail to adequately address the nuances of cross-domain applications, where agents must interact with diverse systems and stakeholders.

Researchers from OpenAI have proposed a comprehensive set of practices designed to enhance the safety and reliability of agentic AI systems, addressing the above shortcomings. These include robust task suitability assessments, where systems are rigorously tested for their capacity to handle specific goals across varying conditions. Another key recommendation involves the imposition of operational constraints, such as limiting agents’ ability to perform high-stakes actions without explicit human approval. Researchers also emphasize the importance of ensuring agents’ behaviors are legible to users by providing detailed logs and reasoning chains. This transparency allows for better monitoring and debugging of agent operations. Also, researchers advocate for designing systems with interruptibility in mind, enabling users to halt operations seamlessly in case of anomalies or unforeseen issues.

The proposed practices rely on advanced methodologies to mitigate risks effectively. For instance, automatic monitoring systems can track agents’ actions and flag deviations from expected behaviors in real-time. These systems utilize classifiers or secondary AI models to analyze and evaluate agent outputs, ensuring compliance with predefined safety protocols. Fallback mechanisms are also critical; these involve predefined procedures that activate if an agent is abruptly terminated. For example, if an agent managing financial transactions is interrupted, it could automatically notify all relevant parties to mitigate disruptions. Also, the researchers stress the need for multi-party accountability frameworks, ensuring developers, deployers, and users share responsibility for preventing harm.

The researchers’ findings demonstrate the effectiveness of these measures. In controlled scenarios, implementing task-specific evaluations reduced error rates by 37%, while transparency measures enhanced user trust by 45%. Agents with fallback mechanisms demonstrated a 52% improvement in system recovery during unexpected failures. When combined with real-time intervention capabilities, automatic monitoring systems achieved a 61% success rate in identifying and correcting potentially harmful actions before escalation. These results underscore the feasibility and benefits of adopting a structured approach to agentic AI governance.

Key takeaways from the research are outlined as follows:

Comprehensive task assessments ensure agents are suited for specific goals, reducing operational risks by up to 37%.
Requiring explicit approvals for high-stakes actions minimizes the likelihood of critical errors.
Detailed logs and reasoning chains improve user trust and accountability by 45%.
Secondary AI systems significantly enhance oversight, achieving a 61% success rate in identifying harmful actions.
Predefined procedures improve system resilience, reducing disruption during unexpected failures by 52%.
Shared responsibility among developers, deployers, and users ensures a balanced risk management approach.

In conclusion, the OpenAI study presents a compelling case for adopting structured safety practices in agentic AI systems. The proposed framework mitigates risks by addressing critical issues such as task suitability, transparency, and accountability while enabling the benefits of advanced AI. These practices offer a practical roadmap for ensuring that agentic AI systems operate responsibly and align with societal values. With measurable improvements in safety and efficiency, this research lays the foundation for widespread, trustworthy deployment of agentic AI systems.

The post OpenAI Researchers Propose Comprehensive Set of Practices for Enhancing Safety, Accountability, and Efficiency in Agentic AI Systems appeared first on MarkTechPost.

Researchers at Stanford Use AI and Spatial Transcriptomics to Discover What Makes Some Cells Age Faster/Slower in the Brain

Sana Hassan — Sun, 22 Dec 2024 03:28:58 +0000

Aging is linked to a significant rise in neurodegenerative diseases like Alzheimer’s and cognitive decline. While brain aging involves complex molecular and cellular changes, our understanding of these processes within their spatial context remains limited. Past studies have provided valuable insights into age-related brain changes at a single-cell level but lack comprehensive spatiotemporal resolution. High-throughput spatial omics offer the potential for uncovering cell interactions during aging, yet current research focuses either on spatial or temporal aspects, not both. Advanced computational tools are urgently needed to analyze spatial omics data, enabling a deeper understanding of cell-type-specific changes and interactions during aging.

Stanford University and UCLA researchers created a spatially resolved single-cell transcriptomics atlas of 4.2 million mouse brain cells spanning 20 age points across the adult lifespan. They also examined the effects of rejuvenating interventions, such as exercise and partial reprogramming. They developed spatial aging clocks using this atlas—machine learning models identifying transcriptomic aging patterns, rejuvenation, and disease. Their findings reveal that T cells have a pro-aging effect on nearby cells, while neural stem cells exert a rejuvenating influence. These insights highlight the significant impact of rare cell types on brain aging and offer potential targets for anti-aging therapies.

The study constructed a comprehensive single-cell transcriptomic atlas of the mouse brain, profiling 4.2 million cells across 20 age points spanning adulthood. This allowed for the creating of spatial aging clocks—machine learning models trained to identify aging, rejuvenation, and disease-related transcriptomic signatures across different brain regions and cell types. The method also considered rare cell populations, enhancing the precision of age-related changes in the brain. By leveraging these spatial clocks, the researchers were able to detect cell-type-specific patterns linked to aging processes, providing a detailed understanding of age-related shifts in brain biology.

In addition, deep learning methods were used to see the role of specific cell types in aging and rejuvenation. The study revealed that T cells infiltrate the brain with age and have a pro-aging impact on neighboring cells, while neural stem cells exert rejuvenating effects on surrounding tissue. These findings were linked to specific molecular mediators, suggesting that targeting certain cell types might effectively combat tissue aging. This highlights the potential for therapeutic strategies to modulate cell interactions in aging brains to promote rejuvenation and reduce age-related decline.

The study developed spatial transcriptomic clocks by analyzing 4.2 million cells across 20 distinct ages in the adult mouse brain. These clocks identified spatial and cell-type-specific transcriptomic signatures associated with aging, rejuvenation, and disease, including those of rare cell types. Notably, T cells, which increase in the brain with age, were found to have a pro-aging effect on neighboring cells. Conversely, neural stem cells exhibited a pro-rejuvenating impact in adjacent cells. The research also identified potential mediators of these effects, offering insights into cellular interactions that influence brain aging.

In conclusion, the study provides a detailed spatial analysis of aging in the mouse brain, enabling the tracking of gene expression changes across regions and cell types. The developed spatial aging clocks can be used to assess the effects of interventions on aging and disease processes at single-cell resolution. The authors highlight the need for further research to understand the mechanisms behind cell proximity effects, particularly in neurons. They suggest that more in-depth studies, including functional assays and deeper imaging, are required to fully elucidate how T cells and neural stem cells influence brain aging and potential therapeutic strategies for enhancing resilience during aging.

The post Researchers at Stanford Use AI and Spatial Transcriptomics to Discover What Makes Some Cells Age Faster/Slower in the Brain appeared first on MarkTechPost.

Can AI Models Scale Knowledge Storage Efficiently? Meta Researchers Advance Memory Layer Capabilities at Scale

Sana Hassan — Sat, 21 Dec 2024 06:48:52 +0000

The field of neural network architectures has witnessed rapid advancements as researchers explore innovative ways to enhance computational efficiency while maintaining or improving model performance. Traditional dense networks rely heavily on computationally expensive matrix operations to encode and store information. This reliance poses challenges when scaling these models for real-world applications that demand extensive knowledge storage and retrieval. Recent research has focused on refining existing architectures to balance computational and memory requirements, providing a pathway for more scalable and energy-efficient AI systems.

The limitations of existing models are their inefficiency in handling simple factual associations, such as relationships between entities or numerical facts. Dense transformer models, while effective in representing complex patterns, require increases in computational resources as their parameter count grows. This inefficiency is problematic when addressing tasks requiring factual accuracy, such as question answering, where the ability to recall specific information is critical. The challenge lies in finding methods that enable models to store and retrieve knowledge without significantly inflating computational demands or memory usage. The need for solutions that scale efficiently with increased parameter size and data demands has become increasingly urgent.

Current techniques, such as mixture-of-experts (MOE) models, have been developed to address some of these challenges. MOE introduces sparsity by activating only a subset of its parameters for a given input, reducing computational overhead compared to fully dense models. However, MOE architectures often fall short in tasks requiring precise factual recall and general knowledge representation. Also, these methods typically require intricate designs and are challenging to implement at scale. Despite this, MOE models have struggled to fully address the growing demands for efficient, scalable architectures, prompting researchers to explore alternative approaches.

To advance the utility of memory layers in AI architectures, researchers from FAIR at Meta focused on scaling and improving their implementation. Initially proposed as a key-value lookup mechanism, memory layers have shown a potential to store and retrieve information efficiently. Meta researchers integrated these memory layers into transformer architectures, replacing feed-forward networks in various configurations. This effort represents a two-order-of-magnitude improvement in memory capacity, with memory parameters scaling up to 128 billion. By revising and optimizing memory layers, the team demonstrated their ability to outperform dense and MOE models in various benchmarks, especially those requiring factual accuracy and knowledge retrieval.

Image Source

The refined memory layer design incorporates trainable key-value embeddings and leverages sparse activation patterns to enhance efficiency. Product-key lookup, a technique that splits keys into smaller subsets for efficient search, enabled the scaling of memory layers without exponential computational growth. Parallel memory operations across GPUs further streamlined performance, allowing the system to handle millions of keys while maintaining a manageable computational load. In earlier implementations, custom CUDA kernels optimized memory operations, achieving GPU bandwidths close to 3 TB/s compared to less than 400 GB/s.

Image Source

In evaluations, for example, a 1.3 billion-parameter model with memory layers achieved comparable accuracy to dense models with twice the computational requirements. In factual question-answering tasks like NaturalQuestions and TriviaQA, memory-augmented models exhibited over a 100% increase in accuracy. Scaling experiments revealed that memory models with 64 million keys and 128 billion memory parameters approached the performance of the Llama2 7B model, which required more computational resources. Also, memory-augmented models showed faster learning rates, reaching high accuracy with fewer training tokens.

Image Source

Several takeaways from the research include:

Memory layers enhanced performance in factual question-answering benchmarks, outperforming dense models with double the computational resources.
The approach scaled seamlessly across parameter sizes, reaching 128 billion memory parameters and demonstrating consistent accuracy improvements.
Custom CUDA kernels maximized GPU bandwidth, ensuring efficient implementation of memory operations.
Memory-augmented models achieved superior results earlier in training, showcasing their ability to learn efficiently with fewer tokens.
Shared memory pools allowed for a strategic blend of dense and memory layers, optimizing computational and memory efficiency.

In conclusion, Meta FAIR’s research advances the scalability and utility of memory layers in AI models. The study underscores the potential for memory layers to address critical challenges in neural network architectures by refining the implementation and demonstrating their efficiency across various tasks. These findings highlight a promising direction, providing tools to balance computational demands with enhanced knowledge storage capabilities.

The post Can AI Models Scale Knowledge Storage Efficiently? Meta Researchers Advance Memory Layer Capabilities at Scale appeared first on MarkTechPost.