Latest AI and tech news

evidentlyai.com • Jan 1 • by Evidently AI

Automation · Languages +12

+ = Sign in to like, bookmark, or love articles.

1776A

LLM EvalsHow we built open-source automated prompt optimizationIn this blog, we explain common prompt optimization strategies and show how we designed and implemented automated prompt optimization in the open-source Evidently Python library.

evidentlyai.com • Dec 22 • by Evidently AI

Machine learning · Generative AI +9

+ = Sign in to like, bookmark, or love articles.

43BF8

CommunityLearnings from 800+ GenAI and ML use casesWe collected and analyzed 800 real-world ML and GenAI use cases from 150+ companies. Here's what we've learned about how companies actually use AI in production and how it has evolved.

evidentlyai.com • Dec 3 • by Evidently AI

Automation · AI +12

+ = Sign in to like, bookmark, or love articles.

42390

LLM EvalsAI risk: 10 pitfalls to avoid when building AI productsIn this blog, we break down ten common AI risks, share actionable ways to mitigate them, and outline a practical approach for building a continuous AI testing workflow.

evidentlyai.com • Nov 27 • by Evidently AI

DevOps & SRE · Web development +12

+ = Sign in to like, bookmark, or love articles.

A4CBA

EvidentlyEvidently 0.7.17: open-source LLM tracing and dataset managementWe released a major update that brings previously closed functionality to the open-source. Evidently Open-source now supports data storage backend, raw dataset management, LLM tracing storage and viewer.

evidentlyai.com • Nov 14 • by Evidently AI

Legal · AI & society +11

+ = Sign in to like, bookmark, or love articles.

F361F

LLM EvalsHow to align LLM judge with human labels: a hands-on tutorialIn this tutorial, we will walk you through the process of designing and testing an LLM evaluator. We will also create an LLM judge that evaluates the quality of LLM-generated code reviews.

evidentlyai.com • Oct 31 • by Evidently AI

Legal · AI +11

+ = Sign in to like, bookmark, or love articles.

1EDB4

LLM EvalsF.A.Q on LLM judges: 7 questions we often getLLM-as-a-judge is a popular approach to testing and evaluating AI systems. We answered some of the most common questions about how LLM judges work and how to use them effectively.

evidentlyai.com • Oct 30 • by Evidently AI

Marketing & sales · AI +10

+ = Sign in to like, bookmark, or love articles.

5AB09

Community7 agentic AI examples and use casesIn this blog, we will explore seven agentic AI examples and use cases in the real world — from transaction analysis to e-commerce recommendations to code review.

evidentlyai.com • Oct 21 • by Evidently AI

AI · Evals +8

+ = Sign in to like, bookmark, or love articles.

A025F

Community25 AI benchmarks: examples of AI models evaluationIn this blog, we’ll explore AI benchmarks and why we need them. We’ll also provide 25 examples of widely used AI benchmarks for reasoning and language understanding, conversation abilities, coding, information retrieval, and tool use.

evidentlyai.com • Oct 8 • by Evidently AI

AI · Evals +10

+ = Sign in to like, bookmark, or love articles.

46134

Community8 AI hallucinations examplesAI hallucinations come in different forms: from giving factually incorrect responses to inventing nonexistent product features or even people. We compiled eight real-life AI hallucination examples.

evidentlyai.com • Sep 5 • by Evidently AI

Databases · Machine learning +10

+ = Sign in to like, bookmark, or love articles.

DE3B0

CommunityGen AI use cases in 2025: learnings from 650+ real-world examplesSince 2023, we've been curating a database of real-world AI and ML use cases. Here is what we've learned from 650+ examples from top companies.

evidentlyai.com • Aug 28 • by Evidently AI

AI economy · AI agents +10

+ = Sign in to like, bookmark, or love articles.

41E52

Community10 AI agents examples from top companiesWe compiled 10 real-world examples of companies using AI agents to transform their business operations: from financial data analysis to customer support to employee productivity.

evidentlyai.com • Aug 22 • by Evidently AI

Automation · AI +11

+ = Sign in to like, bookmark, or love articles.

88A54

LLM EvalsWhat is an LLM evaluation framework? Workflows and tools explained.In this guide, we explain what it means to design an LLM evaluation framework for your AI application and introduce Evidently, an open-source LLM evaluation tool.

evidentlyai.com • Aug 7 • by Evidently AI

Open source · AI +11

+ = Sign in to like, bookmark, or love articles.

4676E

EvidentlyEvidently 0.7.11: open-source synthetic data generation for LLM systemsIf you’re building an AI system, you need solid test data. We made a synthetic data generator so you can create custom test datasets that mimic real-world data. Available in Evidently Open-source.

evidentlyai.com • Jul 11 • by Evidently AI

AI · Evals +10

+ = Sign in to like, bookmark, or love articles.

AB42C

Evidently10 AI agent benchmarksWe put together 10 AI agent benchmarks designed to assess how well different LLMs perform as agents in real-world scenarios, tackling challenges like planning, decision-making, and tool use.

evidentlyai.com • Jul 11 • by Evidently AI

AI coding tools · Evals +9

+ = Sign in to like, bookmark, or love articles.

D7990

Community15 LLM coding benchmarksThis blog highlights 15 LLM coding benchmarks designed to evaluate and compare how different models perform on various coding tasks, including code completion, snippet generation, debugging, and more.

evidentlyai.com • Jul 11 • by Evidently AI

Automation · LLMs +12

+ = Sign in to like, bookmark, or love articles.

AD684

EvidentlyEvidently 0.7.10: open-source prompt optimizer for LLM judgesCrafting a good prompt for an LLM judge isn’t trivial. We built a tool that automates LLM judge prompt generation based on expert-labeled datasets. Available in Evidently Open-source.

evidentlyai.com • Jun 25 • by Evidently AI

Legal · Open source +10

+ = Sign in to like, bookmark, or love articles.

9C798

LLM EvalsIs this email okay? We asked a jury of LLM judges.In this blog tutorial, we walk through the idea of LLM juries, the open-source implementation, and what we learned from LLM disagreements.

evidentlyai.com • Jun 23 • by Evidently AI

Automation · LLMs +13

+ = Sign in to like, bookmark, or love articles.

59C1A

EvidentlyCI/CD for LLM apps: Run tests with Evidently + GitHub actionsWe released a GitHub Action that brings LLM output quality checks to your CI workflow and lets you auto-test your LLM system on every code update. Available in Evidently Open-source and Evidently Cloud.

evidentlyai.com • Jun 20 • by Evidently AI

Generative AI · AI +10

+ = Sign in to like, bookmark, or love articles.

9A4AA

LLM EvalsOWASP Top 10 LLM: How to test your Gen AI app in 2025In this blog, we’ll walk you through the OWASP Top 10 LLM list of vulnerabilities for LLM applications, explore strategies to mitigate these risks, and show how to apply them to keep your AI product safe and reliable.

evidentlyai.com • Jun 5 • by Evidently AI

AI in education · Generative AI +10

+ = Sign in to like, bookmark, or love articles.

D6D17

CommunityFree LLM and Gen AI courses to take in 2025Looking for free LLM and Gen AI courses to attend in 2025? We put together 10 online LLM and Gen AI courses that focus on the practical side of building with LLMs. They are free to join or publish their content for everyone to access without a fee.

evidentlyai.com • May 6 • by Evidently AI

RAG · Evals +9

+ = Sign in to like, bookmark, or love articles.

EED8C

Community7 RAG benchmarksWe highlight seven RAG benchmarks that help measure and compare how well different LLMs handle core RAG challenges like large context windows, grounded reasoning, and using retrieved evidence effectively.

evidentlyai.com • Apr 9 • by Evidently AI

Machine learning · Cloud +10

+ = Sign in to like, bookmark, or love articles.

C8BE5

TutorialsHow to set up ML monitoring with email alerts using Evidently and AWS SESIn this tutorial, you will learn how to implement Evidently checks as part of an ML pipeline and send email notifications based on a defined condition.

evidentlyai.com • Mar 25 • by Evidently AI

LLMs · Evals +7

+ = Sign in to like, bookmark, or love articles.

35B65

CommunityHow companies evaluate LLM systems: 7 examples from Asana, GitHub, and moreWe put together 7 examples of how top companies like Asana and GitHub run LLM evaluations. They share how they approach the task, what methods and metrics they use, what they test for, and their learnings along the way.

evidentlyai.com • Feb 28 • by Evidently AI

Responsible AI · AI safety +9

+ = Sign in to like, bookmark, or love articles.

3A1BD

Community10 LLM safety and bias benchmarksLLM safety benchmarks help to ensure models are robust and reliable. In this blog, we highlight 10 key safety and bias LLM benchmarks that help assess and improve LLM reliability.

evidentlyai.com • Feb 13 • by Evidently AI

RAG · Evals +10

+ = Sign in to like, bookmark, or love articles.

23565

EvidentlyEvidently 0.6.3: Open-source RAG evaluation and testingEvidently open-source now has more tools for evaluating RAG. You can score context relevance, evaluate generation quality, and use different LLMs as evaluators.

evidentlyai.com • Feb 13 • by Evidently AI

RAG · Automation +10

+ = Sign in to like, bookmark, or love articles.

3CE04

LLM Evals10 RAG examples and use cases from real companiesRAG helps make LLM systems more accurate and reliable. We compiled 10 real-world examples of how companies use RAG to improve customer experience, automate routine tasks, and improve productivity.

evidentlyai.com • Jan 24 • by Evidently AI

MLOps · Open source +9

+ = Sign in to like, bookmark, or love articles.

0F844

EvidentlyUpcoming Evidently API ChangesThe Evidently API is evolving — and it’s getting better! We are updating the open-source Evidently API to make it simpler, more flexible, and easier to use. Explore the new features.

evidentlyai.com • Jan 9 • by Evidently AI

Cost of AI · AI policy +9

+ = Sign in to like, bookmark, or love articles.

A39B1

CommunityAI regulations: EU AI Act, AI Bill of Rights, and moreIn this guide, we’ll discuss key AI regulations, such as the EU AI Act and the Blueprint for AI Bill of Rights, and explain what they mean for teams building AI-powered products.

evidentlyai.com • Nov 27 • by Evidently AI

Machine learning · Marketing & sales +10

+ = Sign in to like, bookmark, or love articles.

B0D16

CommunityAMA with Jacopo Tagliabue: reasonable scale ML, testing recommendation systems, and hot DataOpsIn this blog, we recap the Ask-Me-Anything session with Jacopo Tagliabue, where we chatted about ML at a reasonable scale, testing RecSys, MLOps anti-patterns, what’s hot in DataOps, fundamentals in MLOps, and more.

evidentlyai.com • Sep 25 • by Evidently AI

Prompting · Cybersecurity +9

+ = Sign in to like, bookmark, or love articles.

F33E4

CommunityLLM hallucinations and failures: lessons from 5 examplesReal-world examples of LLM hallucinations and other failures that can occur in LLM-powered products in the wild, such as prompt injection and out-of-scope usage scenarios.

evidentlyai.com • Sep 17 • by Evidently AI

Responsible AI · AI +8

+ = Sign in to like, bookmark, or love articles.

184BE

CommunityWhen AI goes wrong: 13 examples of AI mistakes and failuresFrom being biased to making things up, there are numerous instances where we’ve seen AI going wrong. In this post, we’ll explore thirteen notable AI failures when the technology didn’t perform as expected.

evidentlyai.com • Sep 9 • by Evidently AI

Legal · Automation +10

+ = Sign in to like, bookmark, or love articles.

4B452

TutorialsWrong but useful: an LLM-as-a-judge tutorialThis tutorial shows how to create, tune, and use LLM judges. We'll make a toy dataset and assess correctness and verbosity. You can apply the same workflow for other criteria.

evidentlyai.com • Sep 5 • by Evidently AI

Automation · Cloud +12

+ = Sign in to like, bookmark, or love articles.

FE989

EvidentlyMeet Evidently Cloud for AI Product TeamsWe are launching Evidently Cloud, a collaborative AI observability platform built for teams developing products with LLMs. It includes tracing, datasets, evals, and a no-code workflow. Check it out!

evidentlyai.com • Jul 15 • by Evidently AI

LLMs · real-world +5

+ = Sign in to like, bookmark, or love articles.

C9AC0

Community55 real-world LLM applications and use cases from top companiesHow do companies use LLMs in production? We compiled 55 real-world LLM applications from companies that share their learnings from building LLM systems.

evidentlyai.com • Jul 1 • by Evidently AI

Automation · AI coding tools +11

+ = Sign in to like, bookmark, or love articles.

25FD5

TutorialsLLM regression testing workflow step by step: code tutorialIn this tutorial, we introduce the end-to-end workflow of LLM regression testing. You will learn how to run regression testing as a process and build a dashboard to monitor the test results.

evidentlyai.com • Jun 20 • by Evidently AI

Software engineering · LLMs +5

+ = Sign in to like, bookmark, or love articles.

515E1

TutorialsWatch the language: A tutorial on regression testing for LLMs

evidentlyai.com • Jun 19 • by Evidently AI

Machine learning · MLOps +8

+ = Sign in to like, bookmark, or love articles.

44CBB

CommunityMLOps Zoomcamp recap: how to monitor ML models in production?Our CTO Emeli Dral was an instructor for the ML Monitoring module of MLOps Zoomcamp 2024, a free MLOps course. We summarized the ML monitoring course notes and linked to all the practical videos.

evidentlyai.com • Jun 19 • by Evidently AI

Machine learning · Data science +9

+ = Sign in to like, bookmark, or love articles.

37E3B

CommunityAI, Machine Learning, and Data Science conferences to attend in 2025We put together the most interesting conferences on AI, Machine Learning, and Data Science in 2025. And the best part? Some of them are free to attend or publish the content after the event.

evidentlyai.com • May 28 • by Evidently AI

AI assistants · Languages +10

+ = Sign in to like, bookmark, or love articles.

5266C

EvidentlyEvidently 0.4.25: An open-source tool to evaluate, test and monitor your LLM-powered appsEvidently open-source Python library now supports evaluations for LLM-based applications, including RAGs and chatbots. You can compare, test, and monitor your LLM system quality from development to production.

evidentlyai.com • Dec 14 • by Evidently AI

Data engineering · Languages +10

+ = Sign in to like, bookmark, or love articles.

07A8A

Evidently7 new features at Evidently: ranking metrics, data drift on Spark, and moreDid you miss some of the latest updates at Evidently open-source Python library? We summed up a few features we shipped recently in one blog.

evidentlyai.com • Oct 25 • by Evidently AI

Machine learning · Data science +12

+ = Sign in to like, bookmark, or love articles.

2D76C

TutorialsBatch inference and ML monitoring with Evidently and PrefectIn this tutorial, you will learn how to run batch ML model inference and deploy a model monitoring dashboard for production ML models using open-source tools.

evidentlyai.com • Oct 4 • by Evidently AI

Machine learning · Data science +9

+ = Sign in to like, bookmark, or love articles.

D0F18

CommunityMLOps courses to take in 2023Looking for MLOps courses to attend in 2023? We put together five great online MLOps courses for data scientists and ML engineers. They are free to join or publish their content for everyone to access without a fee.