Connect with us

Artificial Intelligence

Snowflake and Meta Partner to Deliver Next-Gen AI

Published

on

Snowflake has announced that it will host the Llama 3.1 collection of multilingual open-source large language models (LLMs) in Snowflake Cortex AI for enterprises to easily harness and build powerful AI applications at scale. This offering includes Meta’s largest and most powerful open-source LLM, Llama 3.1 405B, with Snowflake developing and open-sourcing the inference system stack to enable real-time, high-throughput inference and further democratize powerful natural language processing and generation applications.

Snowflake’s industry-leading AI Research Team has optimized Llama 3.1 405B for both inference and fine-tuning, supporting a massive 128K context window from day one, while enabling real-time inference with up to 3x lower end-to-end latency and 1.4x higher throughput than existing open source solutions. Moreover, it allows for fine-tuning on the massive model using just a single GPU node — eliminating costs and complexity for developers and users — all within Cortex AI.

By partnering with Meta, Snowflake is providing customers with easy, efficient, and trusted ways to seamlessly access, fine-tune, and deploy Meta’s newest models in the AI Data Cloud, with a comprehensive approach to trust and safety built-in at the foundational level.

“Snowflake’s world-class AI Research Team is blazing a trail for how enterprises and the open source community can harness state-of-the-art open models like Llama 3.1 405B for inference and fine-tuning in a way that maximizes efficiency,” said Vivek Raghunathan, VP of AI Engineering, Snowflake. “We’re not just bringing Meta’s cutting-edge models directly to our customers through Snowflake Cortex AI. We’re arming enterprises and the AI community with new research and open source code that supports 128K context windows, multi-node inference, pipeline parallelism, 8-bit floating point quantization, and more to advance AI for the broader ecosystem.”

Snowflake’s AI Research Team continues to push the boundaries of open-source innovations through its regular contributions to the AI community and transparency around how it is building cutting-edge LLM technologies. In tandem with the launch of Llama 3.1 405B, Snowflake’s AI Research Team is now open-sourcing its Massive LLM Inference and Fine-Tuning System Optimization Stack in collaboration with DeepSpeed, Hugging Face, vLLM, and the broader AI community. This breakthrough establishes a new state-of-the-art for open source inference and fine-tuning systems for multi-hundred billion parameter models.

Massive model scale and memory requirements pose significant challenges for users aiming to achieve low-latency inference for real-time use cases, high throughput for cost-effectiveness, and long context support for various enterprise-grade generative AI use cases. The memory requirements of storing model and activation states also make fine-tuning extremely challenging, with the large GPU clusters required to fit the model states for training often inaccessible to data scientists.

Snowflake’s Massive LLM Inference and Fine-Tuning System Optimization Stack addresses these challenges. By using advanced parallelism techniques and memory optimizations, Snowflake enables fast and efficient AI processing, without needing complex and expensive infrastructure. For Llama 3.1 405B, Snowflake’s system stack delivers real-time, high-throughput performance on just a single GPU node and supports massive 128k context windows across multi-node setups.

This flexibility extends to both next-generation and legacy hardware, making it accessible to a broader range of businesses. Moreover, data scientists can fine-tune Llama 3.1 405B using mixed precision techniques on fewer GPUs, eliminating the need for large GPU clusters. As a result, organizations can adapt and deploy powerful enterprise-grade generative AI applications easily, efficiently, and safely.

Snowflake’s AI Research Team has also developed optimized infrastructure for fine-tuning inclusive of model distillation, safety guardrails, retrieval augmented generation (RAG), and synthetic data generation so that enterprises can easily get started with these use cases within Cortex AI.

AI safety is of the utmost importance to Snowflake and its customers. As a result, Snowflake is making Snowflake Cortex Guard generally available to further safeguard against harmful content for any LLM application or asset built in Cortex AI — either using Meta’s latest models, or the LLMs available from other leading providers including AI21 Labs, Google, Mistral AI, Reka, and Snowflake itself. Cortex Guard leverages Meta’s Llama Guard 2, further unlocking trusted AI for enterprises so they can ensure that the models they’re using are safe.

Artificial Intelligence

Check Point Leverages AI to Strengthen Network Security

Published

on

Check Point Software Technologies has announced the new Check Point Quantum Firewall Software R82 (R82) and additional innovations for the Infinity Platform. As organizations face a 75% surge in cyber-attacks worldwide, R82 delivers new AI-powered engines to prevent zero-day threats including phishing, malware, and domain name system (DNS) exploits. It also includes new architectural changes and innovations that drive DevOps agility for data centre operations as well as simplicity and scale.

“Threats are continuing to multiply exponentially, and organizations need intelligent solutions that can keep them a step ahead,” said Nataly Kremer, Chief Product Officer at Check Point Software Technologies. “Network security is increasingly strategic. Our suite of AI-powered threat prevention tools – from Check Point Quantum Firewall Software R82 to GenAI Protect and more – are not only bringing world-class innovations but also relentlessly focused on making it operationally simple and resilient.”

Quantum Software R82 delivers over 50 new capabilities for enterprise customers including:

  • Industry Leading AI-Powered Threat Prevention to block 99.8% of zero-day threats. It introduces four new AI engines to find hidden relationships and patterns to block over 500K additional attacks per month that protect against sophisticated zero-day phishing and malware campaigns.
  • Agile Datacenter Operations to accelerate app development with automated integration of security policy.
  • With dramatically simplified firewall virtualization, organizations achieve 3X faster provisioning of virtual systems for multi-tenancy and agile application development benefiting DevOps.
  • Operational Simplicity to offer seamless scalability for networks of all sizes, automatically adapting to business growth and traffic spikes. It enables organizations to achieve resilience with built-in load sharing and clustering technology (ElasticXL) while benefiting from 3x faster provisioning and operations for firewall management.
  • Post-Quantum Cryptography (PQC) to provide the latest NIST-approved cryptography Kyber (ML-KEM) for quantum-safe encryption, assuring that today’s encrypted data won’t turn into tomorrow’s treasure chest for threat actors.

“Maintaining effective network security requires AI, automation, and the ability to adapt quickly to the latest threats,” said Frank Dickson, IDC Group Vice President of Security and Trust. “Security needs to be strong, but it also needs to enable business innovation at the speed of DevOps. With Check Point’s new collaborative AI-powered solutions and Quantum Firewall Software, Check Point looks to deliver high-performance AI threat prevention while enabling organizations to innovate quickly.”

The new capabilities build upon Check Point’s recently released suite of AI-powered threat prevention innovations:

  1. Check Point Infinity AI Copilot is a responsive AI-powered assistant designed to automate and accelerate security management and threat resolution.
  2. Check Point GenAI Protect is a pioneering solution for the safe adoption of generative AI in enterprises.
  3. Check Point Infinity External Risk Management (ERM) delivers continuous monitoring and real-time threat prevention, augmented by expert-managed services. This protects customers against a wider array of external risks, from credential threat and vulnerability exploitation to phishing attacks and fraud.

“We’ve seen a definite performance increase and operational value with our upgrade to Check Point’s Quantum Firewall Software R82 software release. The new Quantum Firewall software allows us to secure and manage our encrypted traffic more easily than ever,” said Jeff Burgess, Manager of I.T. Enterprise, Aviation Technical Services. “With Check Point, all of our security products are working in sync together to provide a level of security which was previously unattainable.”

Continue Reading

Artificial Intelligence

Dataiku Launches LLM Guard Services to Control Generative AI Rollouts

Published

on

Dataiku has announced the launch of its LLM Guard Services suite, designed to advance enterprise GenAI deployments at scale from proof-of-concept to full production without compromising cost, quality, or safety. Dataiku LLM Guard Services includes three solutions: Cost Guard, Safe Guard, and the newest addition, Quality Guard. These components are integrated within the Dataiku LLM Mesh, the market’s most comprehensive and agnostic LLM gateway, for building and managing enterprise-grade GenAI applications that will remain effective and relevant over time. LLM Guard Services provides a scalable no-code framework to foster greater transparency, inclusive collaboration, and trust in GenAI projects between teams across companies.

Today’s enterprise leaders want to use fewer tools to reduce the burden of scaling projects with siloed systems, but 88% do not have specific applications or processes for managing LLMs, according to a recent Dataiku survey. Available as a fully integrated suite within the Dataiku Universal AI Platform, LLM Guard Services is designed to address this challenge and mitigate common risks when building, deploying, and managing GenAI in the enterprise.

“As the AI hype cycle follows its course, the excitement of two years ago has given way to frustration bordering on disillusionment today. However, the issue is not the abilities of GenAI, but its reliability,” said Florian Douetteau, Dataiku CEO. “Ensuring that GenAI applications deliver consistent performance in terms of cost, quality, and safety is essential for the technology to deliver its full potential in the enterprise. As part of the Dataiku Universal AI platform, LLM Guard Services is effective in managing GenAI rollouts end-to-end from a centralized place that helps avoid costly setbacks and the proliferation of unsanctioned ‘shadow AI’ – which are as important to the C-suite as they are for IT and data teams.”

Dataiku LLM Guard Services provides oversight and assurance for LLM selection and usage in the enterprise, consisting of three primary pillars:

  • Cost Guard: A dedicated cost-monitoring solution to enable effective tracing and monitoring of enterprise LLM usage to anticipate better and manage spend vs. budget of GenAI.
  • Safe Guard: A solution that evaluates requests and responses for sensitive information and secures LLM usage with customizable tooling to avoid data abuse and leakage.
  • Quality Guard: The newest addition to the suite that provides quality assurance via automatic, standardized, code-free evaluation of LLMs for each use-case to maximize response quality and bring both objectivity and scalability to the evaluation cycle.

Previously, companies deploying GenAI have been forced to use custom code-based approaches to LLM evaluation or leverage separate, pure-play point solutions. Now, within the Dataiku Universal AI Platform, enterprises can quickly and easily determine GenAI quality and integrate this critical step in the GenAI use-case building cycle. By using LLM Quality Guard, customers can automatically compute standard LLM evaluation metrics, including LLM-as-a-judge techniques like answer relevancy, answer correctness, context precision, etc., as well as statistical techniques such as BERT, Rouge and Bleu, and more to ensure they select the most relevant LLM and approach to sustain GenAI reliability over time with greater predictability. Further, Quality Guard democratizes GenAI applications so any stakeholder can understand the move from proof-of-concept experiments to enterprise-grade applications with a consistent methodology for evaluating quality.

Continue Reading

Artificial Intelligence

Cloudflare Helps Content Creators Regain Control of their Content from AI Bots

Published

on

Cloudflare has announced AI Audit, a set of tools to help websites of any size analyse and control how their content is used by artificial intelligence (AI) models. For the first time, website and content creators will be able to quickly and easily understand how AI model providers are using their content, and then take control of whether and how the models can access it. Additionally, Cloudflare is developing a new feature where content creators can reliably set a fair price for their content that is used by AI companies for model training and retrieval augmented generation (RAG).

Website owners, whether for-profit companies, media and news publications, or small personal sites, may be surprised to learn AI bots of all types are scanning their content thousands of times every day without the content creator knowing or being compensated, causing significant destruction of value for businesses large and small. Even when website owners are aware of how AI bots are using their content, they lack a sophisticated way to determine what scanning to allow and a simple way to take action. For society to continue to benefit from the depth and diversity of content on the Internet, content creators need the tools to take back control.

“AI will dramatically change content online, and we must all decide together what its future will look like,” said Matthew Prince, co-founder and CEO, Cloudflare. “Content creators and website owners of all sizes deserve to own and have control over their content. If they don’t, the quality of online information will deteriorate or be locked exclusively behind paywalls. With Cloudflare’s scale and global infrastructure, we believe we can provide the tools and set the standards to give websites, publishers, and content creators control and fair compensation for their contribution to the Internet, while still enabling AI model providers to innovate.”

With AI Audit, Cloudflare aims to give content creators information and take back control so there can be a transparent exchange between the websites that want greater control over their content, and the AI model providers that are in need of fresh data sources so that everyone benefits. With this announcement, Cloudflare aims to help any website:

  • Automatically control AI bots, for free: AI is a quickly evolving space, and many website owners need time to understand and analyze how AI bots are affecting their traffic or business. Many small sites don’t have the skills or bandwidth to manually block AI bots. The ability to block all AI bots in one click puts content creators back in control.
  • Tap into analytics to see how AI bots access their content: Every site using Cloudflare now has access to analytics to understand why, when, and how often AI models access their website. Website owners can now make a distinction between bots – for example, text-generative bots that still credit the source of the data they use when generating a response, versus bots that scrape data with no attribution or credit.
  • Better protect their rights when negotiating with model providers: An increasing number of sites are signing agreements directly with model providers to license the training and retrieval of content in exchange for payment. Cloudflare’s AI Audit tab will provide advanced analytics to understand metrics that are commonly used in these negotiations, like the rate of crawling for certain sections or the entire page. Cloudflare will also model terms of use that every content creator can add to their sites to legally protect their rights.
  • Set a fair price for the right to scan content and transact seamlessly (in development): Many site owners, whether they are the large companies of the future or a high-quality individual blogs, do not have the resources, context, or expertise to negotiate one-off deals that larger publishers are signing with AI model providers, and AI model providers do not have the bandwidth to do this with every site that approaches them. In the future, even the largest content creators will benefit from Cloudflare’s seamless price setting and transaction flow, making it easy for model providers to find fresh content to scan they may otherwise be blocked from, and content providers to take control and be paid for the value they create.
Continue Reading
Advertisement

Follow Us

Trending

Copyright © 2021 Security Review Magazine. Rysha Media LLC. All Rights Reserved.