This post was originally published on this site Meta’s Llama 3.1 and Mistral Large 2 open-source AI models challenge industry leaders, reshaping the AI landscape and democratizing access to cutting-edge technology.Read More

In the domain of sequential decision-making, especially in robotics, agents often deal with continuous action spaces and high-dimensional observations. These difficulties result from making decisions across a broad range of potential actions like complex, continuous action spaces and evaluating enormous volumes of data. Advanced procedures are needed to process and act upon the information in these scenarios in an efficient and effective manner.

In recent research, a team of researchers from the University of Maryland, College Park, and Microsoft Research has presented a new viewpoint that formulates the problem of sequence compression in terms of creating temporal action abstractions. Large language models’ (LLMs) training pipelines are the source of inspiration for this method in the field of natural language processing (NLP). Tokenizing input is a crucial part of LLM training, and it’s commonly accomplished using byte pair encoding (BPE). This research suggests adapting BPE, which is commonly utilized in NLP, to the task of learning variable timespan abilities in continuous control domains.

Primitive Sequence Encoding (PRISE) is a new approach which has been introduced by the research to put this theory into practice. PRISE produces efficient action abstractions by fusing BPE and continuous action quantization. In order to facilitate processing and analysis, continuous activities are quantized by converting them into discrete codes. These discrete code sequences are then compressed using the BPE sequence compression technique to reveal significant and recurrent action primitives.

Empirical studies use robotic manipulation tasks to show the effectiveness of PRISE. The study has demonstrated that the high-level skills identified improve behavior cloning’s (BC) performance on downstream tasks through the use of PRISE on a series of multitask robotic manipulation demonstrations. Compact and meaningful action primitives produced by PRISE are useful for Behaviour Cloning, an approach where agents learn from expert examples.

The team has summarized their primary contributions as follows.

Primitive Sequence Encoding (PRISE), a unique method for learning multitask temporal action abstractions using NLP approaches, is the main contribution of this work. 

To simplify the action representation, PRISE converts the continuous action space of the agent into discrete codes. These distinct action codes are arranged in a sequence based on pretraining trajectories. These action sequences are used by PRISE to extract skills with varied timesteps.

PRISE considerably improves learning efficiency over strong baselines such as ACT by learning policies over the learned skills and decoding them into simple action sequences during downstream tasks.

Research involves in-depth research to comprehend how different parameters affect PRISE’s performance, demonstrating the vital function BPE plays in the project’s success.

In conclusion, temporal action abstractions present a potent means of improving sequential decision-making when seen as a sequence compression problem. Through the effective integration of NLP approaches, particularly BPE, into the continuous control domain, PRISE is able to learn and encode high-level skills. These abilities show the promise of interdisciplinary approaches in increasing robotics and artificial intelligence, in addition to enhancing the effectiveness of techniques such as behavior cloning.

Check out the Paper and Project. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter..

Don’t Forget to join our 47k+ ML SubReddit

Find Upcoming AI Webinars here

Large Language Models (LLMs) face deployment challenges due to latency issues caused by memory bandwidth constraints. Researchers use weight-only quantization to address this, compressing LLM parameters to lower precision. This approach improves latency and reduces GPU memory requirements. Implementing this effectively requires custom mixed-type matrix-multiply kernels that move, dequantize, and process weights efficiently. Existing kernels like bits and bytes, Marlin, and BitBLAS have shown significant speed-ups but are often limited to 4-bit quantization. Recent advancements in odd-bit and non-uniform quantization methods highlight the need for more flexible kernels that can support a wider range of settings to maximize the potential of weight quantization in LLM deployment.

Researchers have attempted to solve the LLM deployment challenges using weight-only quantization. Uniform quantization converts full-precision weights to lower-precision intervals, while non-uniform methods like lookup table (LUT) quantization offer more flexibility. Existing kernels like bits and bytes, Marlin, and BitBLAS move quantized weights from main memory to on-chip SRAM, performing matrix multiplications after de-quantizing to floating-point. These show significant speed-ups but often specialize in 4-bit uniform quantization, with LUT-quantization kernels underperforming. Non-uniform methods like SqueezeLLM and NormalFloat face trade-offs between lookup table size and quantization granularity. Also, non-uniformly quantized operations can’t utilize GPU accelerators optimized for floating-point calculations. This highlights the need for efficient kernels that can utilize quantized representations to minimize memory movement and GPU-native floating-point matrix multiplications, balancing the benefits of quantization with hardware optimization.

Researchers from Massachusetts Institute of Technology, High School of Mathematics Plovdiv and Carnegie Mellon University, MBZUAI, Petuum Inc. introduce an innovative approach that,  flexible lookup-table engine (FLUTE) for deploying weight-quantized LLMs, focusing on low-bit and non-uniform quantization. It addresses three main challenges: handling sub-8-bit matrices, optimizing lookup table-based dequantization, and improving workload distribution for small batches and low-bit-width weights. FLUTE overcomes these issues through three key strategies: offline weight restructuring, a shared-memory lookup table for efficient dequantization, and Stream-K partitioning for optimized workload distribution. This approach enables FLUTE to effectively manage the complexities of low-bit and non-uniform quantization in LLM deployment, improving efficiency and performance in scenarios where traditional methods fall short.

FLUTE is an innovative approach for, flexible mixed-type matrix multiplications in weight-quantized LLMs. It addresses key challenges in deploying low-bit and non-uniform quantized models through three main strategies:

Offline Matrix Restructuring: FLUTE reorders quantized weights to optimize for Tensor Core operations, handling non-standard bit widths (e.g., 3-bit) by splitting weights into bit-slices and combining them in registers.

Vectorized Lookup in Shared Memory: To optimize dequantization, FLUTE uses a vectorized lookup table stored in shared memory, accessing two elements simultaneously. It also employs table duplication to reduce bank conflicts.

Stream-K Workload Partitioning: FLUTE implements Stream-K decomposition to evenly distribute workload across SMs, mitigating wave quantization issues in low-bit and low-batch scenarios.

These innovations allow FLUTE to efficiently fuse dequantization and matrix multiplication operations, optimizing memory usage and computational throughput. The kernel employs a sophisticated pipeline of data movement between global memory, shared memory, and registers, utilizing GPU hardware capabilities for maximum performance in weight-quantized LLM deployments.

FLUTE shows impressive performance across various matrix shapes on both A6000 and A100 GPUs. On the A6000, it occasionally approaches the theoretical maximum speedup of 4x. This performance is also consistent across different batch sizes, unlike other LUT-compatible kernels which typically achieve similar speedups only at a batch size of 1 and then degrade rapidly as batch size increases. Also, FLUTE’s performance compares well even to Marlin, a kernel highly specialized for FP16 input and uniform-quantized INT4 weights. This demonstrates FLUTE’s ability to efficiently handle both uniform and non-uniform quantization schemes.

FLUTE demonstrates superior performance in LLM deployment across various quantization settings. The learned NF quantization approach outperforms standard methods and combines well with AWQ. FLUTE’s flexibility allows for experiments with different bit widths and group sizes, nearly matching 16-bit baseline perplexity with small group sizes. End-to-end latency tests using vLLM framework showed meaningful speedups across various configurations, including with Gemma-2 models. A group size of 64 was found to balance quality and speed effectively. Overall, FLUTE proves to be a versatile and efficient solution for quantized LLM deployment, offering improved performance across multiple scenarios.

FLUTE is a CUDA kernel designed to accelerate LLM inference through fused quantized matrix multiplications. It offers flexibility in mapping quantized to de-quantized values via lookup tables and supports various bit widths and group sizes. FLUTE’s performance is demonstrated through kernel-level benchmarks and end-to-end evaluations on state-of-the-art LLMs like LLaMA-3 and Gemma-2. Tested on A6000 and A100 GPUs in single and tensor parallel setups, FLUTE shows efficiency across unquantized, 3-bit, and 4-bit configurations. This versatility and performance make FLUTE a promising solution for accelerating LLM inference using advanced quantization techniques.

Check out the Paper and GitHub. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter..

Don’t Forget to join our 47k+ ML SubReddit

Find Upcoming AI Webinars here

This post was originally published on this site Salesforce’s MINT-1T, a trillion-token dataset, revolutionizes AI research by democratizing access and raising crucial ethical questions about the future of machine learning.Read More

Large Language Models (LLMs) have revolutionized the field of natural language processing, allowing machines to understand and generate human language. These models, such as GPT-4 and Gemini-1.5, are crucial for extensive text processing applications, including summarization and question answering. However, managing long contexts remains challenging due to computational limitations and increased costs. Researchers are, therefore, exploring innovative approaches to balance performance and efficiency.

A notable challenge in processing lengthy texts is the computational burden and associated costs. Traditional methods often need to improve when dealing with long contexts, necessitating new strategies to handle this issue effectively. This problem requires methodologies that balance high performance with cost efficiency. One promising approach is Retrieval Augmented Generation (RAG), which retrieves relevant information based on a query and prompts LLMs to generate responses within that context. RAG significantly expands a model’s capacity to access information economically. However, a comparative analysis becomes essential with advancements in LLMs like GPT-4 and Gemini-1.5, which show improved capabilities in directly processing long contexts.

Researchers from Google DeepMind and the University of Michigan introduced a new method called SELF-ROUTE. This method combines the strengths of RAG and long-context LLMs (LC) to route queries efficiently using model self-reflection to decide whether to use RAG or LC based on the nature of the query. The SELF-ROUTE method operates in two steps. Initially, the query and retrieved chunks are provided to the LLM to determine if the query is answerable. If deemed answerable, the RAG-generated answer is used. Otherwise, the LC will be given the full context for a more comprehensive response. This approach significantly reduces computational costs while maintaining high performance, effectively leveraging the strengths of both RAG and LC models.

The SELF-ROUTE evaluation involved three recent LLMs: Gemini-1.5-Pro, GPT-4, and GPT-3.5-Turbo. The study benchmarked these models using LongBench and u221eBench datasets, focusing on query-based tasks in English. The results demonstrated that LC models consistently outperformed RAG in understanding long contexts. For example, LC surpassed RAG by 7.6% for Gemini-1.5-Pro, 13.1% for GPT-4, and 3.6% for GPT-3.5-Turbo. However, RAG’s cost-effectiveness remains a significant advantage, particularly when the input text considerably exceeds the model’s context window size.

SELF-ROUTE achieved notable cost reductions while maintaining comparable performance to LC models. For instance, the cost was reduced by 65% for Gemini-1.5-Pro and 39% for GPT-4. The method also showed a high degree of prediction overlap between RAG and LC, with 63% of queries having identical predictions and 70% showing a score difference of less than 10. This overlap suggests that RAG and LC often make similar predictions, both correct and incorrect, allowing SELF-ROUTE to leverage RAG for most queries and reserve LC for more complex cases.

The detailed performance analysis revealed that, on average, LC models surpassed RAG by significant margins: 7.6% for Gemini-1.5-Pro, 13.1% for GPT-4, and 3.6% for GPT-3.5-Turbo. Interestingly, for datasets with extremely long contexts, such as those in u221eBench, RAG sometimes performed better than LC, particularly for GPT-3.5-Turbo. This finding highlights RAG’s effectiveness in specific use cases where the input text exceeds the model’s context window size.

The study also examined various datasets to understand the limitations of RAG. Common failure reasons included multi-step reasoning requirements, general or implicit queries, and long, complex queries that challenge the retriever. By analyzing these failure patterns, the research team identified potential areas for improvement in RAG, such as incorporating chain-of-thought processes and enhancing query understanding techniques.

In conclusion, the comprehensive comparison of RAG and LC models highlights the trade-offs between performance and computational cost in long-context LLMs. While LC models demonstrate superior performance, RAG remains viable due to its lower cost and specific advantages in handling extensive input texts. The SELF-ROUTE method effectively combines the strengths of both RAG and LC, achieving performance comparable to LC at a significantly reduced cost.

Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter..

Don’t Forget to join our 47k+ ML SubReddit

Find Upcoming AI Webinars here

This post was originally published on this site A new research by Google DeepMind shows how sparse autoencoders (SAEs) with special JumpReLU activation can help interpret LLMs.Read More

This post was originally published on this site 75% of knowledge workers are already using AI at work, and that usage has doubled in the past six months.Read More

This post was originally published on this site Uncategorized Published at: July 26, 20242 views In the evolving landscape of artificial intelligence, a controversial niche has emerged: undress AI tools. These software applications, which use advanced algorithms to simulate the removal of clothing in images, have sparked debates about privacy, ethics, and the boundaries of

Artificial intelligence, particularly in training large multimodal models (LMMs), relies heavily on vast datasets that include sequences of images and text. These datasets enable the development of sophisticated models capable of understanding and generating multimodal content. As AI models’ capabilities advance, the need for extensive, high-quality datasets becomes even more critical, driving researchers to explore new data collection and curation methods.

A significant challenge in AI research is the need for large-scale, open-source, multimodal interleaved datasets. These datasets are essential for training models seamlessly integrating text and image data. The limited availability of such datasets hampers the development of robust and high-performing open-source models, resulting in a performance gap between open-source and proprietary models. Addressing this gap requires innovative approaches to dataset creation that can provide the necessary scale and diversity.

Existing methods for creating multimodal datasets often involve collecting and curating data from HTML documents. Notable datasets like OBELICS have been instrumental but are limited in scale and diversity, primarily sourcing data from HTML. This restriction affects the variety and richness of the data, impacting the performance and applicability of the resulting AI models. Researchers have found that datasets sourced solely from HTML documents must capture the full spectrum of required multimodal content for comprehensive model training.

Researchers from the University of Washington, Salesforce Research, Stanford University, the University of Texas at Austin, and the University of California Berkeley introduced MINT-1T, the most extensive & diverse open-source multimodal interleaved dataset to date, addressing the need for larger and more varied datasets. MINT-1T comprises one trillion text tokens and 3.4 billion images from HTML, PDFs, and ArXiv papers. This dataset represents a tenfold increase from previous datasets, significantly enhancing the data for training multimodal models. Institutions such as the University of Washington and Salesforce Research collaborated on this initiative, demonstrating a concerted effort to bridge the gap in dataset availability.

Creating the MINT-1T dataset involved an intricate process of sourcing, filtering, and deduplicating data. HTML documents were expanded to include data from earlier years, and PDFs were processed to extract readable text and images. ArXiv papers were parsed for figures and text, ensuring a comprehensive collection of multimodal content. Advanced filtering methods were employed to remove low-quality, non-English, and inappropriate content. Deduplication processes were also implemented to eliminate repetitive data, ensuring the dataset’s quality and diversity.

Experiments demonstrated that LMMs trained on the MINT-1T dataset matched and often surpassed the performance of models trained on previous leading datasets like OBELICS. Including more diverse sources in MINT-1, T resulted in better generalization and performance across various benchmarks. Notably, the dataset significantly improved performance in tasks involving visual question answering and multimodal reasoning. The researchers found that models trained on MINT-1T performed better across multiple demonstrations, highlighting the dataset’s effectiveness.

The MINT-1T dataset’s construction included detailed steps to ensure data quality and diversity. For instance, the dataset consists of 922 billion HTML tokens, 106 billion PDF tokens, and 9 billion ArXiv tokens. The filtering process involved eliminating documents with inappropriate content and non-English texts, using tools like Fasttext for language identification and NSFW detectors for image content. The deduplication process was crucial, involving Bloom filters to remove duplicate paragraphs and documents and hashing techniques to eliminate repetitive images.

In conclusion, the MINT-1T dataset addresses dataset scarcity and diversity. By introducing a larger and more varied dataset, the researchers have enabled the development of more robust and high-performing open-source multimodal models. This work highlights the importance of data diversity and scale in AI research and paves the way for future improvements and applications in multimodal AI. The dataset’s extensive scale, including one trillion text tokens and 3.4 billion images, provides a solid foundation for advancing AI capabilities.

Check out the Paper, Details, and GitHub. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter..

Don’t Forget to join our 47k+ ML SubReddit

Find Upcoming AI Webinars here